PaperlayerView exact revision on arXiv

Source-linked AI summary

Manipulation in Clutter with Whole-Arm Tactile Sensing

Advait Jain, Marc D. Killpack, Aaron Edsinger, Charles C. Kemp

arXiv:1304.6146v1cs.RO

TL;DR

Reaching through clutter is challenging when geometry and object mechanics are unknown, and non-contact paths may be unavailable. The paper presents a haptic controller for whole-arm contact, reporting successful reaching with low forces and better performance than a baseline. Its results are subject to limitations in the linear-spring contact model.

  • Problem

    Reaching in clutter motivates methods that can operate using touch when vision and audition are unavailable.

    Source: A. Biological Inspiration · p. 2

  • Method

    A horizon-one MPC controller uses a linear quasi-static model and haptic sensing to regulate forces across multiple contacts on the arm.

    Source: §I · p. 1, §VIII · p. 17

  • Results

    Across simulated and real-robot tests, the controller reached goals in high clutter with low forces and outperformed a baseline; in simulation it succeeded in 150% more trials, with average contact forces of 5.9N vs. 28.6N.

    Source: §VIII · p. 17

  • Takeaways & Limitations

    The experiments show that robots can reach goals in clutter while making arm-wide contact and pushing aside or deforming objects.

    Source: §I · p. 1, §VIII · p. 17

    The linear-spring contact model is computationally favorable but predicts adhesive forces when breaking contact.

    Source: C. Limitations · pp. 16–17

Abstract

from arXiv · show

We begin this paper by presenting our approach to robot manipulation, which emphasizes the benefits of making contact with the world across the entire manipulator. We assume that low contact forces are benign, and focus on the development of robots that can control their contact forces during goal-directed motion. Inspired by biology, we assume that the robot has low-stiffness actuation at its joints, and tactile sensing across the entire surface of its manipulator. We then describe a novel controller that exploits these assumptions. The controller only requires haptic sensing and does not need an explicit model of the environment prior to contact. It also handles multiple contacts across the surface of the manipulator. The controller uses model predictive control (MPC) with a time horizon of length one, and a linear quasi-static mechanical model that it constructs at each time step. We show that this controller enables both real and simulated robots to reach goal locations in high clutter with low contact forces. Our experiments include tests using a real robot with a novel tactile sensor array on its forearm reaching into simulated foliage and a cinder block. In our experiments, robots made contact across their entire arms while pushing aside movable objects, deforming compliant objects, and perceiving the world.

I. INTRODUCTION

This introduction argues that whole-arm contact can help robots move through clutter, provided contact forces remain low. It motivates whole-body tactile sensing and compliant actuation, and reports improved reaching outcomes with the proposed controller.

I. INTRODUCTION

  • Unlike approaches that treat unintended contact as failure, the paper treats sufficiently low-force contact as benign and focuses on exploiting it.The assumed penalty-free force range depends on the robot and environment.Source: §I · p. 1
  • The approach assumes compliant joint actuation to reduce contact forces and tactile sensing across the arm to monitor force magnitude and distribution.Source: §I · p. 1
  • The controller achieved higher success rates and lower contact forces than a baseline across simulated and real-robot tests.Robots also compressed, bent, or moved objects aside while reaching.Source: §I · p. 1

A. Biological Inspiration

  • Biological examples motivate touch-based manipulation, including operation without vision and methods that degrade gracefully when non-haptic senses are unavailable.The paper notes touch’s role in foraging and cites animals that manipulate without vision.Source: A. Biological Inspiration · pp. 1–2

B. Benefits of Whole-body Contact and Tactile Sensing

  • Allowing arm contact can increase effective range of motion, especially when the robot pushes movable objects or compresses compliant ones.Avoiding contact, applying safety margins, and being unable to move objects all reduce available motion.Source: B. Benefits of Whole-body Contact and… · p. 2, Figure 3 · p. 2
  • Joint-mounted force sensing can leave multi-contact configurations ambiguous, while tactile sensing across the arm can distinguish contact patterns and their force distributions.The distribution and geometry of contact can affect damage risk and the robot’s subsequent movement options.Source: B. Benefits of Whole-body Contact and… · pp. 2–3

C. Challenges Associated with Reaching in High Clutter

Reaching in high clutter is difficult when non-contact paths are scarce, objects are unfamiliar or occluded, and their mechanics are unknown. The paper proposes haptic, whole-arm contact control and positions it against prior clutter-manipulation work.

C. Challenges Associated with Reaching in High Clutter

  • Clutter can eliminate non-contact paths, make sequential end-effector clearing inefficient, and contain unique, occluded objects with mechanics that are difficult to infer without contact.Such environments may include fixed, movable, rigid, deformable, granular, fluid-like, or dynamic objects.Source: C. Challenges Associated with Reaching… · p. 3

D. Our Approach

  • The approach permits simultaneous contacts across the arm, requires no detailed environment model before contact, and uses haptic sensing and arm kinematics to estimate contact geometry and stiffness.The controller assumes initial task and robot parameters, including a tolerated force magnitude and an initial stiffness estimate for new contacts.Source: D. Our Approach · pp. 3–4

E. Organization of this Paper

A. Manipulation in Clutter

  • Prior clutter-manipulation work often restricts contact to the end effector and uses pre-existing models; this work targets haptic, whole-arm reaching and evaluates diverse environments.The cited similar approach by Mason et al. (2011) grasps markers from clutter and uses durable, rigid, nearly identical objects.Source: A. Manipulation in Clutter · p. 4

B. Multi-contact Manipulation

The paper contrasts its multi-contact controller with approaches that rely on detailed environmental or object models, rigid stationary contacts, or long-horizon planning. It applies a linear quasi-static model and MPC to manipulation with multiple unpredicted contacts, while drawing parallels to contact-rich robot locomotion.

B. Multi-contact Manipulation

  • Earlier multi-contact frameworks require a full dynamic model, stationary rigid contacts, and at least six degrees of freedom per contact; reported tests did not cover unpredicted contacts or lost contacts.The cited results were in simulation or relatively controlled real-robot settings.Source: B. Multi-contact Manipulation · p. 4
  • On a real robot with tactile skin, the controller handles multiple unpredicted contacts with compliant, rigid, movable, and fixed obstacles across the arm, without requiring six degrees of freedom per contact.The controller uses a linear quasi-static model of the robot’s interaction with the environment.Source: B. Multi-contact Manipulation · p. 4
  • Prior humanoid planning uses predetermined contacts and complete geometric models, assumes rigid stationary contacts, and does not incorporate sensor feedback during execution.Those approaches address balance, whereas this paper assumes a statically stable robot.Source: B. Multi-contact Manipulation · pp. 4–5

C. Motion Planning with Deformable Objects

  • Prior deformable-object planners require knowledge of object configuration and accurate deformation models, which can be difficult or infeasible to obtain before contact in realistic clutter.The challenge is heightened when multiple objects touch and their configurations are unknown.Source: C. Motion Planning with Deformable… · p. 5

D. Robot Locomotion

  • Robot locomotion offers a parallel: robots have traversed clutter without detailed terrain models or long-horizon planning, using whole-body contact and simple mechanical models.The cited locomotion work also uses compliance and force sensing.Source: D. Robot Locomotion · p. 5

E. Model Predictive Control

  • MPC has been used in chemical process control, aerial vehicles, robot locomotion, and robot manipulators; in aerial-vehicle control it is often called receding-horizon control.The cited work establishes prior use across several control settings, not this paper’s specific manipulation approach.Source: E. Model Predictive Control · p. 5

A. Overview of the One-Step Model Predictive Controller

The controller uses a one-step linear MPC to move the end effector toward a goal while constraining predicted contact forces. Its model combines compliant joints with spring-like contacts and is rebuilt from tactile and joint measurements at each time step.

A. Overview of the One-Step Model Predictive Controller

  • The one-step MPC moves the end effector toward a goal subject to predicted contact-force constraints, allowing forces below a don’t-care threshold without penalty.It also limits predicted force changes and stops updating the virtual trajectory if a safety threshold is exceeded.Source: A. Overview of the One-Step Model… · pp. 5–6

B. Control Structure

  • A 1 kHz joint-space impedance controller runs beneath an outer MPC loop at 50–100 Hz, which uses encoder and tactile feedback to update the virtual trajectory.The inner controller is described as “simple” impedance control.Source: B. Control Structure · p. 6

C. Linear Discrete-Time Model

  • The per-step linear model assumes quasi-static dynamics, perfect gravity compensation, frictionless contacts, and linear springs acting normal to the arm at contacts.It combines joint-space motion with contact forces and estimates contact force, location, and stiffness from whole-arm tactile sensing and joint encoders.Source: C. Linear Discrete-Time Model · p. 7

D. Quadratic Program to Compute ∆φ∗

The controller solves a quadratic program over the incremental change in virtual trajectory. The objective tracks a desired end-effector motion, while linear constraints bound joint angles and contact forces.

D. Quadratic Program to Compute ∆φ∗

  • The quadratic objective selects the virtual-trajectory increment that brings predicted hand motion closest to the desired motion toward the goal.The desired increment follows a straight line toward the goal and uses a small-distance step when the goal is farther away.Source: D. Quadratic Program to Compute ∆φ∗ · pp. 7–8
  • Linear inequalities keep predicted joint angles within physical joint limits by expressing the bounds in terms of the virtual-trajectory increment.The current configuration determines the difference between the current angles and their minimum and maximum limits.Source: D. Quadratic Program to Compute ∆φ∗ · p. 8
  • For each contact, constraints bound force below its don’t-care threshold and limit its predicted change over one time step, with no added cost below the threshold.The resulting inequalities are expressed in the optimization variable ∆φ.Source: D. Quadratic Program to Compute ∆φ∗ · p. 8

E. Extensions to the Quadratic Program

The paper adds three quadratic-program extensions for its experiments: limiting torque changes, reducing forces above the don’t-care threshold, and respecting virtual-trajectory limits on one robot.

E. Extensions to the Quadratic Program

  • A squared-magnitude torque-change term discourages large joint-torque changes in one time step and helps prevent large motions of redundant degrees of freedom.The term is added to the objective with a scalar weight.Source: E. Extensions to the Quadratic Program · p. 8
  • When a contact force exceeds its don’t-care threshold, modified constraints prevent its predicted increase and an added objective term encourages the force to decrease.The desired force change has constant magnitude and points opposite to the current contact force.Source: E. Extensions to the Quadratic Program · p. 8
  • For Cody, additional linear constraints keep the virtual trajectory within the physical joint limits enforced by its impedance controller.These constraints are placed on the virtual-trajectory increment.Source: E. Extensions to the Quadratic Program · p. 8

IV. EXPERIMENTAL TESTBEDS

The paper evaluates its controller in three testbeds, using the same MPC code across them. The software simulation provides a planar arm with whole-arm tactile sensing interacting with fixed and movable obstacles.

IV. EXPERIMENTAL TESTBEDS

  • The evaluation uses software simulation, hardware-in-the-loop skin simulation with a real 7 DoF arm, and a real 7 DoF arm with forearm tactile sensors.The same MPC code, written in Python, runs on all three testbeds.Source: §IV · p. 8

A. Software Simulation

  • The software testbed simulates a three-joint planar arm with tactile skin covering its entire surface at 100 taxels per meter.The arm has a 1 kHz joint-space impedance controller and is simulated with Open Dynamics Engine.Source: A. Software Simulation · p. 8
  • The simulated arm’s torso, shoulder, and elbow interact with cylindrical obstacles that are either fixed or movable.Movable obstacles can slide when applied force exceeds about 2 N; fixed obstacles remain stationary.Source: A. Software Simulation · pp. 8–9

B. The Robot

Cody’s hardware supports real-arm experiments, while a forearm tactile array and a hardware-in-the-loop testbed extend sensing and controller evaluation. Contact-force and location measurements differ across the testbeds.

B. The Robot

  • Cody has two compliant 7 DoF arms, but its tactile skin currently covers only the right forearm.The forearm skin experiments use only the arm’s first four degrees of freedom because a wrist cover supports distal contact sensing.Source: B. The Robot · p. 9

C. Real Tactile Skin Sensor

  • The forearm skin has 384 taxels in a 16 × 24 array, with 9 mm × 9 mm elements and a 0–30 N sensing range.Data are available at 100 Hz; two sleeve layers cover open joints, protect the sensor, and make the arm exterior low friction.Source: C. Real Tactile Skin Sensor · p. 9

D. Hardware-in-the-loop Skin Simulation

  • Because Cody lacks whole-arm skin, the hardware-in-the-loop testbed estimates contacts on a real arm using tracked geometry and instrumented obstacles.It estimates at most one contact location per link-obstacle pair; when multiple links contact one obstacle, the measured force magnitude is divided equally among them.Source: D. Hardware-in-the-loop Skin Simulation · pp. 9–10

E. Tactile Feedback in the Different Testbeds

  • Contact force and location have different operational meanings across the software, hardware-in-the-loop, and real-skin testbeds.Software uses simulated taxel-normal forces; hardware-in-the-loop uses link-level force estimates without simulated taxels or normal components; real-skin measurements combine taxel-normal forearm forces with a wrist force-torque measurement distally.Source: E. Tactile Feedback in the Different… · p. 10

F. Low Stiffness at the Joints

The experiments use low-stiffness joints and compare tactile MPC with a tactile-safety baseline and a planner that knows the environment. Reaching trials can retry after a failed attempt, though pull-out failures affected some simulated trials.

F. Low Stiffness at the Joints

  • The robot and simulated arm use impedance control with low joint stiffness, including simulated proximal-to-distal stiffnesses of 30, 20, and 15 Nm/rad.Cody uses different settings, and its final two wrist joints are held stiff apart from passive compliance.Source: F. Low Stiffness at the Joints · p. 10

V. APPROACHES USED FOR COMPARISON

  • The comparisons include a tactile-feedback baseline and a geometric motion planner with full environment knowledge.The planner estimates optimal success rates in software simulation.Source: §V · p. 10

A. Baseline Controller

  • The baseline shares the MPC’s low-stiffness impedance control but does not use tactile feedback except for a safety stop.In free space, with no joint limits and only the goal-distance objective, the controllers produce the same straight-line end-effector motion.Source: A. Baseline Controller · pp. 10–11

B. Motion Planner

  • The bidirectional RRT planner has complete knowledge of clutter and ignores movable obstacles, which are removed for its comparisons.Its estimated optimal success rate may be an overestimate because remaining movable obstacles could block a solution.Source: B. Motion Planner · p. 11

VI. EXPERIMENTS

  • Experiments evaluate three arm-and-sensing configurations and compare reaching against the planner and baseline in selected trials.Contact force and location mean different things across the testbeds.Source: §VI · p. 11

A. Pull Out and Retry

  • An additional control layer can pull the arm out, reposition or reconfigure it, and retry when the end effector stops moving.Pulling out caused some failures in software simulation, but none in the cited real-arm experiments.Source: A. Pull Out and Retry · p. 11

B. Software Simulation Testbed

In software simulation, MPC outperformed the baseline on success rate and kept contact forces lower. The experiments also show that the force-threshold parameter predictably influences contact forces.

B. Software Simulation Testbed

  • Evaluation compared estimated optimal success with baseline and MPC across 2,420 trials spanning fixed and movable obstacle counts from 0 to 20.The trials varied the two obstacle counts in steps of two.Source: B. Software Simulation Testbed · p. 11
  • Across 100 trials, median force and f_thresh_ci had correlation coefficient ≥0.998; changing this parameter predictably influenced contact forces.Fig. 12 reports contact-force medians and quartiles across threshold settings.Source: Figure 12 · p. 12, B. Software Simulation Testbed · p. 12
  • MPC single reach exceeded baseline success by 48.1 percentage points (157.7% increase); up to six reaches added 12.5 points and kept forces lower.The estimated optimal success rate was 7.1 percentage points above MPC with multiple reaches.Source: B. Software Simulation Testbed · pp. 11–12

C. Hardware-in-the-loop Skin Simulation Testbed

Hardware-in-the-loop tests show that the controller can reach goals while regulating contact forces, including selectively across regions and while pushing into a deformable object. In a five-goal comparison, it outperformed the baseline in success and mean contact force.

C. Hardware-in-the-loop Skin Simulation Testbed

  • Across five hardware-in-the-loop goals, MPC reached all five; the baseline failed two, with mean contact force 3.3N versus 8.1N.The baseline was run from the starting conditions where MPC had succeeded.Source: Table II · p. 13, C. Hardware-in-the-loop Skin… · p. 13
  • Online stiffness estimation let the robot push into a pillow more aggressively and reach the goal faster than using a conservative static stiffness estimate.Stiffness was estimated from the slope relating normal force to contact motion along the contact normal, fit by least squares.Source: C. Hardware-in-the-loop Skin… · p. 13
  • Setting the fragile-region threshold to 2N versus 5N elsewhere let MPC selectively regulate contact forces across regions.The force histograms show the selective control result.Source: C. Hardware-in-the-loop Skin… · p. 13, Figure 15 · p. 14

D. Forearm Tactile Skin Sensor

Forearm tactile-skin experiments evaluate reaching in clutter, including foliage and a cinder-block opening. The discussion interprets the results as promising for reaching into unknown environments, while noting that broader evaluation remains necessary.

D. Forearm Tactile Skin Sensor

  • Because tactile skin covered only the forearm, experiments excluded contacts on the elbow and upper arm; the foliage combined compliant leaves with rigid, fixed wooden blocks.Leaves could be pushed aside with relatively low force, while the blocks could not.Source: D. Forearm Tactile Skin Sensor · p. 14
  • One trial each demonstrated reaching in foliage and through a cinder-block opening, whose sharp, abrasive edges surrounded a 14.5cm opening and a 10cm forearm.The cinder block represented a rigid, heavy, fixed object encountered in rubble.Source: D. Forearm Tactile Skin Sensor · p. 14
  • In foliage, MPC reached goals 1, 3, and 5, while the baseline reached only goal 5; MPC kept forces near 5N while the baseline exceeded the 5N safety threshold.The comparison used five automatically generated goals and 20 reach attempts per controller.Source: Table III · p. 14, D. Forearm Tactile Skin Sensor · pp. 14–15

A. The Big Picture

  • The authors judge the approach well-matched to real-world high-clutter manipulation, while emphasizing that further evaluation is required.Demonstrated settings included vegetation with hidden wooden objects, a constrained cinder block, and rigid posts covered with compliant materials; the robot and environments were unscathed in those experiments.Source: A. The Big Picture · pp. 15–16
  • The results suggest greedy feedback can perform well without detailed environment models or long-horizon planning.The authors present this as a suggestion from their system's performance, not as a settled general result.Source: A. The Big Picture · pp. 15–16
  • The authors expect comparable capabilities to benefit service and assistive robots in cluttered human environments and constrained spaces.Examples include reaching into drawers, retrieving objects from under furniture, and helping someone get out of bed.Source: A. The Big Picture · p. 16
  • Tactile feedback lets the arm pivot around a compliant object when force rises beyond the desired level, producing motion that appears sensible without explicitly planned trajectories.The controller regenerates a model from tactile sensing at each time step and greedily chooses its motion.Source: A. The Big Picture · p. 16

B. Future Work

The paper identifies controller limitations and future development needs, then summarizes evaluations in simulation and on real robots. Its conclusion reports reaching in clutter with low contact forces, using compliant actuation and tactile sensing across the arm.

B. Future Work

  • Future work includes higher-level control for local minima, variable joint stiffness, and methods to initialize and adapt controller parameters.The authors suggest data-driven methods as one possible direction for parameter adaptation.Source: B. Future Work · p. 16

C. Limitations

  • The linear-spring contact model is computationally favorable but predicts adhesive forces when contact breaks; actual forces can also exceed the predicted-force constraint.The controller currently responds to such exceedances by removing the constraint and adding a quadratic penalty, making the objective vary over time.Source: C. Limitations · p. 17
  • The quasi-static controller ignores dynamics and is well-matched to slow motion; modeling dynamics might improve higher-speed performance and velocity control.The paper tested end-effector position goals, not posture or full-pose objectives.Source: C. Limitations · p. 17

VIII. CONCLUSION

  • The approach assumes low-stiffness joint actuation and tactile sensing across the arm; its controller uses one-step MPC with a linear quasi-static model.It requires no detailed environment model before contact and handles multiple contacts across the arm.Source: §VIII · p. 17
  • In 2,420 simulated trials, MPC succeeded in 150% more trials than the baseline and had lower average contact forces (5.9N vs. 28.6N).With up to six reach attempts, the greedy controller came within approximately 8% of optimal performance.Source: §VIII · p. 17
  • With real forearm skin in foliage, MPC reached 3 of 5 targets versus 1 of 5 for baseline, with lower average maximum force (5.5N vs. 14.5N).The targets were automatically generated and not necessarily achievable; the robot also reached into a cinder block with rigid, sharp, coarse materials and constrained passages.Source: §VIII · p. 17
  • With simulated skin, the real robot reached five human-specified targets versus three for baseline, with lower average maximum force (5.6N vs. 17.7N).The same tests demonstrated applying less force to a designated fragile region and online estimation of low contact stiffness.Source: §VIII · p. 17

IX. SUPPLEMENTARY MATERIALS

  • Supplementary materials list experiment videos; code and instructions for reproducing software-simulation results were promised if the paper was accepted.The listed videos cover controller comparisons, online stiffness estimation, foliage and cinder-block reaches, and simulated trials.Source: §IX · p. 17
Loading 1304.6146v1…