How Firm Should a Grasp Be?

1 Gentle Grasping Requires Material Perception

• Gentle grasping seeks to minimize potential damage by applying no more force than needed to complete the desired interaction.Excess force can bruise fruit, dent cardboard, or crack or scratch plastic, and repeated careless interactions can cause cumulative wear.Source: §1 · p. 1
• The method uses ephemeral haptic signals during grip and lift to refine visual class-level material priors and adjust force in situ, avoiding time-consuming exploratory probing.The grasp stages include observe, approach, grip, and lift; approach assumes a suitable pose from an existing planner.Source: §1 · pp. 1–2, Figure 1 · p. 2

2 Related Work

• Squash pairs 3D scans with measured properties for 270 fruit and vegetable samples across 52 classes, capturing variation between instances of the same class.Class-level property distributions derived from the dataset reduce the search space for instance-level estimates.Source: §1 · p. 2, §2 · p. 3

1 Gentle Grasping Requires Material Perception

• More than 2× reduction in excess applied force for successful grasps was achieved versus non-adaptive baselines.The visuotactile model ran at 250 Hz on a robot with a parallel-jaw gripper and GelSight sensors.Source: §1 · p. 2

2 Related Work

• Prior gentle-grasping methods include force regulation, tactile feedback, vision, and learned policies, but the cited approaches may lack real-time material inference or use heuristic force predictions.The section also notes that purely vision-based methods have been shown to be more effective when fused with tactile sensing.Source: §2 · p. 3

• Soft objects can require less grasp force than rigid ones because deformation enlarges contact area, adding τAreal shear support to resist gravity.Source: §3 · p. 3, Figure 2 · p. 3
• Theoretical grasp-force reduction is greatest for light, pliable objects, with the plotted mass-to-friction ratio varied while the object and τ stay fixed.Source: Figure 2 · p. 3
• The equilibrium is µF + τAreal = mg, with τ denoting interfacial shear strength and Areal the asperity-interaction area.Source: §3 · p. 3
• When either surface is soft, Areal≈Aapp and tactile sensing observes Aapp, but force and area remain coupled.Source: §3 · p. 3
• Squash supplies measured meshes and material data for 270 produce instances across 52 classes, including property variation for physically accurate simulation.Source: Figure 3 · p. 4
• Rather than rerunning FEA as changing property estimates make updates expensive, the method uses a Hertzian contact approximation and learns (E, m, µ, τ) from temporal haptics to servo-control force.Source: §3 · p. 4

4 Squash: A Dataset of 3D Object Assets and their Physical Properties

• Squash contains 270 produce instances across 52 classes, with measured elastic modulus, mass, volume, and surface friction.The dataset captures variation within object classes.Source: §4 · p. 4
• Per-object measurements combine mass, hardness, scan-derived volume, compression-based elasticity, and slip-onset friction tests across surface locations.Multiple friction trials provide different couplings between applied force and contact area, allowing the friction parameters to be decoupled.Source: §4 · pp. 4–5

5 Material-Aware Robotic Grasping

• The model updates class-conditioned material estimates from tactile images and applied load, then tracks the posterior expected minimum force in real time.As the object lifts, shear and slip update mass and friction estimates; tactile changes during gripping update elasticity.Source: §5 · pp. 5–6
• Simulated minimum grasp force correlated 0.94 with empirical force, and 200k parallel-jaw grasp sequences supplied model training data.The sequences were synthesized using Squash meshes and material-property distributions.Source: §4 · p. 4, §5 · p. 6

Evaluation via Simulations

• In simulation, material-adaptive control outperformed baselines, with force estimates approximately unbiased and closest to ideal; property correlations were rE=0.65, rm=0.88, rµ=0.28, and rτ=0.57.The comparison used 5k simulated objects with high material variability.Source: Table 1 · p. 7, §6 · p. 7
• Removing prior conditioning or the τA contact-model term caused small performance degradations; priors were beneficial but not necessary, and τA improved force accuracy.The limited degradation from prior ablation suggests possible generalization beyond fruits and vegetables.Source: §6 · p. 7

Grasping Real Fruits and Vegetables

• During real grasps, force and material estimates converged quickly; elasticity was refined during grip, while mass and friction were updated during lift.For a nectarine, this enabled force that never significantly exceeded the optimal force.Source: §6 · p. 7, Figure 6 · p. 8
• Across 29 real fruits and vegetables, excess-force distributions clustered more tightly around zero than the baselines while grasp success remained high.The figure describes accurate force control over all 29 tested objects.Source: §6 · pp. 7, 9, Figure 6 · p. 8
• Real-time elasticity estimates enabled sorting fruits by ripeness, which is correlated with hardness in climacteric fruits.The paper presents this as a potential application beyond force control.Source: §6 · p. 9

Qualitative Damage Evaluation

• In internal apple imaging and next-day banana images, only the proposed approach showed no visible damage; the baselines left signs of damage.Internal imaging used fast separation immediately after force application.Source: §6 · p. 9, Figure 7 · p. 9, Figure 8 · p. 9

7 Limitations and Directions for Future Work

• The method minimizes force for a suitable grasp pose; choosing poses that themselves require less force remains unaddressed.The authors give picking up a banana by one end as an example of a pose that may require large forces.Source: §7 · p. 9
• Validation used a parallel-jaw gripper; extending the approach to dexterous hands requires further grasp-physics analysis.Source: §7 · p. 9
• Performance depends on tactile sensing, whose current spatial and temporal resolution trade off; the authors used interpolation and call for sensors with both high update rate and spatial resolution.Source: §7 · p. 9
• The work focuses on fruits and vegetables with measured class priors; effectiveness on objects without such priors remains to be validated.Source: §7 · p. 9
• The method assumes material properties are constant across an object's surface; spatially varying properties remain future work.Source: §7 · p. 9

8 Conclusion

• The visuotactile approach estimates instance-level material properties during contact and uses them to drive closed-loop control toward the theoretical minimum grasp force.Source: §8 · p. 9
• Squash provides 3D object models with measured material properties, and the trained model recovers elasticity, mass, and friction parameters from tactile feedback.Source: §8 · p. 9
• In simulation and on real hardware, the method consistently achieved gentle, non-damaging grasps; in-situ estimates also open avenues for sorting, grasp-pose optimization, and dexterous manipulation.Source: §8 · p. 9

S1 Methods Used to Collect the Squash Dataset

• Structured-light scans from 8–36 views, repeated after reorientation, were postprocessed into watertight meshes; scan volume and kitchen-scale mass were used to calculate density.The scanner's single-shot accuracy was ≤0.05 mm.Source: S1 Methods Used to Collect the Squash… · p. 17
• Elastic modulus was estimated from force and tactile deformation during uniaxial compression, averaging measurements and calibrating the fingertip against a rigid standard.The reported modulus depends on an assumed Poisson ratio, which the setup cannot directly measure.Source: S1 Methods Used to Collect the Squash… · p. 17
• Static friction µ and interfacial shear strength τ were estimated by reducing force on a lifted object until tactile optical flow detected slip, then fitting a linear model across grasp locations.Applied force and contact area at slip were used with the soft friction model in Eq. (1).Source: S1 Methods Used to Collect the Squash… · p. 17

S2.1 Physics Engine and Constitutive Model

• UIPC simulates deformable objects and GelSight-like gel pads with a Stable Neo-Hookean model, tetrahedral FEM, and barrier-based contact; peach probing tests illustrate elasticity and friction measurements.The simulation uses a 1 mm contact threshold and 4 ms time step; probing measures elastic modulus against Poisson ratio and friction coefficient alongside shear strength.Source: S2.1 Physics Engine and Constitutive… · p. 17, Figure S1 · p. 18

S2.2 Scene Configuration

• Scene setup fixes gel-pad properties to match the sensor, samples object properties from class-fitted Squash distributions, assumes ν=0.48, and varies object orientation and dimensions.Contact friction and compliance are calculated using the harmonic mean and Hertz effective modulus, respectively.Source: S2.2 Scene Configuration · p. 18

S2.3 Simulated Gripper Control

• The adaptive gripper closes until contact detection or a displacement limit, then uses proportional control to drive load toward a target before lifting 2 mm.Lifting starts after both jaws meet the servo and force-error criteria, or after 100 servo steps; the sequence is shown in Fig. S2.Source: S2.3 Simulated Gripper Control · pp. 18–19, Figure S2 · p. 19

S2.4 Sensor Observation

• At every time step, the simulator records jaw contact force, force vectors, contact area, centroid, and displacement alongside tactile images.The tactile images include force, pressure, and contact-area maps.Source: S2.4 Sensor Observation · p. 19
• The recorded tactile maps also capture shear, slip, indentation, and binary contact, with each sequence stored alongside its object properties and grip force.Pressure and force maps are rendered in the pad’s image space.Source: S2.4 Sensor Observation · pp. 19–20

S2.5 Parallel and Distributed Execution

• A manifest partitions combinations of object mesh, material properties, and grip force into chunks dispatched to GPU subprocesses for parallel data collection.Multiple grasps are tiled in one scene to amortize GPU setup overhead.Source: S2.5 Parallel and Distributed Execution · p. 20

S2.6 IPC Simulation Sensitivity

• ≤5% MAE change across tested solver settings, but larger friction regularization or time steps reduce μ and τ identifiability.Per-object target forces shift by about 0.1 N; force estimates change −5% when friction regularization is increased tenfold, −4% when the contact barrier is halved or doubled, and +3% when the time step is halved.Source: S2.6 IPC Simulation Sensitivity · p. 20

S3.1 Model Architecture

• A causal CfC sequence encoder processes tactile-image and control features, while a likelihood head scores candidate material parameters from the resulting hidden state.The hidden state depends only on past and present observations, supporting online inference.Source: S3.1 Model Architecture · pp. 20–21
• Online importance sampling reweights 512 fixed prior samples at each time step; the posterior expected minimum force sets the target, and the posterior mean supplies reported material estimates.The prior also serves as the proposal distribution, so prior/proposal terms cancel in normalized importance weights.Source: S3.1 Model Architecture · p. 21

S3.2 Model Training from Precomputed Simulations

• Training uses 200k simulated sequences and a hybrid objective combining InfoNCE comparisons against prior and in-batch negatives with an MSE regression term weighted λ=0.1.The model is trained for 500 epochs with Adam at lr=10−4.Source: S3.2 Model Training from Precomputed… · pp. 21–22

Error Metrics

• Evaluation metrics include bias, MAE, WAPE, MAPE, overload, excess work, excess impulse, and success rate after lifting.Overload measures peak force above the optimal force; excess work and impulse indicate damage risk.Source: S4 Additional Experimental Results · p. 22

Results and Ablations in Simulation

• 83.0% of cases favored the visuotactile model for grasp-force accuracy, versus 15.4% for Class Prior and 1.6% for Naïve; it also achieved lower force than baselines while maintaining gentle grasps in prior ablations.The simulation comparison covers 5k produce items with high variability; Table S1 includes removal and misidentification of the class prior.Source: S4 Additional Experimental Results · p. 22, Figure S3 · p. 23, Table S1 · p. 24

Results in Real-World Experiments

• Real-world estimation errors were low on average but degraded when material properties were far from the class prior, as illustrated across the test-set property range.The reported real-object experiments describe successful grasps with forces intended to minimize damage risk.Source: S4 Additional Experimental Results · p. 22, Figure S4 · p. 23

S5 Robot Hardware and Operating System

• A UR5 arm carries a WSG50-110 parallel-jaw gripper with load cells on both jaws and GelSight Mini tactile sensors providing 8 MP images at 25 Hz.The gripper has a 110 mm maximum opening and 0.1 mm spatial resolution.Source: S5 Robot Hardware and Operating System · p. 24
• A ROS node runs the visuotactile model at 250 Hz, interpolates tactile images to the control rate, and sends target force to a PID gripper loop.The desktop control computer is PTP-synchronized with the Raspberry Pi load-cell recorder.Source: S5 Robot Hardware and Operating System · p. 24