PaperlayerView exact revision on arXiv

Source-linked AI summary

Tactus: Tactile & Pose Sensing Glove for Natural Dexterity Data Collection

Li Guan, Rachael Granberry, Sahar Aseeri, Sahar Rostami, Taylor Niehues, Sai Dhanooshi Marri, Mia Huang, Dingtian Zhang, Ziyi Kou, Jason Pursel, Eric Whitmire, Vatsal Mehta, Ashish Deshpande, Tristan Trutna, Reed Belcher, Emelia Black, Julian Blake, Roger Boldu, Sean Bonaventure, Einar Boyovich, Jordan Brennan, Elizabeth Casillas, David Christensen, Tyler Christensen, Nicholas Colonnese, Stuart Dealey, Gaige DeHaven, Katie Doran, Chris Doughty, Robert Drury, Silvia Espino, Gabriel Geyne, Beth Goodrich, Kyra Gould, Joe Grisso, Susana Guerrero, Andrew Gwozdziowski, Monica Ha, Ehsan Hajiesmaili, Kate Healy, Rose Hope, Michael Hurst, Claire Ikemiya, Daisy Isibor, Rick James, Yashasvi Joginapally, Samuel Johnson, Tyler Jorgensen, Kristy Jost, Ben Kamath, Slava Karulin, Alexander Keener, Woo Kim, Alison Krauskopf, Ankit Kumar, Joshua Lang, Lizzy Li, Cameron Lynch, Daniel Ma, Hector Martinon, Christina Matos, Keagan McCurdy, Lee Mincey, Stuart Nezlek, Tiffany Nguyen, James O'Hara, Jason Osborne, Jacky Pai, Roham Pardakhtim, David Perek, Alexandre Poulin, Nick Raines, Salwa Rasheed, Leah Resneck, Kevin Rhodes, Ergys Ristani, Douglas Roberts, Anik Sarker, Harrison Scarborough, Jonas Schmidtler, Brian Shieh, Benjamin Shuman, Garrett Simard, Forrest Smith, Udara Somarathna, Eliot Sosin, Jonathan Stewart, Kristine Strom, Adam Swett, Hal Talbot-Mantooth, Julia Tanner, Sai Sharan Thirunagari, Curtis Torrey, Simge Uzun, Eileen Wachs, Rylan Wasson, Henry Wei, Jesse White, Beau Woodbyrne, Antonio Woods, Noah Young, Chandler Yount, Joseph Zhang, Sean Keller

arXiv:2610.39076v1cs.ROcs.AIcs.ARcs.LG

TL;DR

Naturalistic manipulation demonstrations matter for robot learning, but vision cannot recover high-fidelity hand pose under occlusion or directly measure contact forces, while exoskeletons can constrain finger motion and alter grasp strategies. Tactus combines textile-integrated force and pose sensing with calibration and standardized benchmarking; its evaluations report 0.59 N minimum detectable force and 11.0 mm root-aligned fingertip tracking error.

  • Problem

    Vision-based methods cannot recover high-fidelity hand pose under occlusion or directly measure contact forces, while exoskeletons can constrain finger motion and alter grasp strategies.

    Source: §I · p. 1

  • Method

    Tactus is a 3D-knit glove with 47 capacitive textile force-sensing locations and 16 6-DoF IMUs, paired with a documented calibration pipeline and standardized benchmarking protocols.

    Source: §I · p. 2, §VII · p. 8

  • Results

    0.59 N minimum detectable force at 95% confidence and 11.0 mm root-aligned fingertip tracking error from offline trajectory optimization; task time was 106% of bare-hand task completion time.

    Source: B. Protocol Tactile · p. 6, §VII · p. 8

  • Takeaways & Limitations

    The benchmarking protocols offer a transferable route from incomparable single-platform demonstrations toward reproducible, quantitative evaluation of sensing gloves.

    Source: §VII · p. 8

Abstract

from arXiv · show

Collecting naturalistic human manipulation demonstrations with accurate contact force and hand pose is essential for robot dexterous manipulation policy and foundational world model learning - yet no existing wearable platform provides both calibrated fingertip force sensing and occlusion-resilient hand tracking in a form factor that preserves natural manipulation behavior. We present Tactus, a textile-integrated sensing glove that co-optimizes three design pillars: (1) a thin, conformal 3D-knit form factor (1.5 mm fingertip stackup) designed to minimize interference with natural manipulation, (2) strategically placed capacitive tactile sensors at manipulation-relevant regions informed by empirical contact studies, and (3) IMU-fused pose tracking that maintains accuracy through the hand-object occlusions where manipulation data is most valuable. Tactus includes a documented calibration pipeline for cross-glove consistency and per-session calibration for subject-dependent correction. To enable objective comparison, we introduce standardized benchmarking protocols for tactile force regression and hand pose estimation, and apply them to Tactus and two commercially available tactile gloves - to our knowledge the first quantitative cross-platform evaluation of this kind. Tactus resolves a minimum detectable force of 0.59 N at 95% confidence, 1.6 - 2.7x lower than the commercial baselines, and with 0.87 cross-participant consistency, while our pose tracking sustains 11.0 mm fingertip keypoint relative error during manipulation, when occlusion is heaviest. We show that these combined signals support object property estimation, slip detection, and representative recording of hand-object interaction - yielding demonstration data with the fidelity required for contact-aware manipulation policies and world models.

I. INTRODUCTION

Naturalistic manipulation demonstrations need both reliable hand pose and contact-force data, yet existing approaches trade natural movement against measurement quality. Tactus combines textile-integrated tactile sensing and pose tracking with a thin glove, calibrated force sensing, and shared benchmarks.

I. INTRODUCTION

  • Vision loses high-fidelity hand pose under manipulation occlusion, while exoskeletons constrain motion; no existing glove jointly provides calibrated force, occlusion-resilient tracking, and validated natural grasp behavior.Source: §I · pp. 1–2
  • Tactus is a textile-integrated glove carrying 47 capacitive force-sensing locations from fingertip to palm and 16 6-DoF IMUs.Source: §I · p. 2
  • A 1.5 mm fingertip stackup and low-profile, stretchable sensors are designed to minimize encumbrance while preserving natural manipulation fidelity.Source: §I · p. 2
  • Tactile placement is informed by ink studies, while force calibration combines scalable factory calibration with a per-session online procedure.Source: §I · p. 2
  • Protocol-Tactile and Protocol-Pose provide standardized benchmarks applied to Tactus and commercial gloves for quantitative cross-platform comparison.Source: §I · p. 2
  • The authors validate Tactus data for object-material classification, slip detection, and multisensory hand-object manipulation.Source: §I · p. 2

II. RELATED WORK

Prior approaches to manipulation capture trade naturalism, direct measurement, and robustness to occlusion. Related work spans vision-based inference, wearable sensing, pose tracking, and tactile calibration, with each leaving performance or validation gaps.

II. RELATED WORK

  • Vision methods infer contact or force from imagery and proxies –, but do not provide calibrated, occlusion-invariant worn measurements.Source: §II · p. 2
  • Wearable gloves offer broad coverage or fingertip force sensing, but calibration and documented real-task performance remain limited; exoskeletons constrain the grasp behavior they measure –,.Source: §II · p. 2
  • Vision tracking degrades during hand-object occlusion, while glove pose sensing is occlusion-invariant but subject to modality-specific errors and inconsistent validation –.Source: §II · pp. 2–3
  • Soft-glove force calibration is perturbed by fit, grounding, and pose; prior approaches use online calibration or reference-free learning, while uniform pneumatic calibration had not addressed soft 3D glove curvature,, –[40].Source: §II · p. 3

III. SYSTEM DESIGN

Tactus is designed around thickness, stiffness, and friction to limit changes to natural manipulation. Its thin integrated construction and conformal fit support task performance and keep sensors coupled to the hand during contact.

Thickness

  • At 1.5 mm fingertip thickness, Tactus reached task completion times similar to barehand performance and was best among tested sensing gloves; thicker configurations increased task time and prompted technique adaptations.Source: §III · p. 3

Tactile sensing

  • The knitted sensor forms a capacitor from conductor and insulation layers, with capacitance changing under external compression.Source: Figure 3 · p. 3

Stiffness

  • Conformal, low-stiffness fit keeps sensors coupled to the hand, whereas oversized gloves can contact the environment before the hand; Tactus reports 0.20 N/cm bending and 0.22 N/cm tensile stiffness per 25 mm width.Source: §III · p. 3, Figure 2 · p. 3

Friction

  • TPU treatment gives Tactus fingertip regions a 0.4 static coefficient of friction on polished steel, balancing grasp stability with the micro-slips used in in-hand manipulation.Source: §III · p. 3

A. Tactile Sensing Design

Tactus combines embedded capacitive textile force sensors with IMU-fused pose tracking. Its sensor layout balances contact coverage against hand geometry and stretch requirements, while the pose system uses distributed IMUs to complement vision during occlusion.

A. Tactile Sensing Design

  • The capacitive pressure sensor is a low-profile double-layer weft knit embedded directly into the textile, with capacitance varying under normal force.Each cell pairs a half-gauge-knit upper electrode with an interlock lower layer.Source: A. Tactile Sensing Design · pp. 3–4
  • Contact mapping, flexed-hand inking, and 3D scans set sensor targets while excluding self-contact folds and palmar concavities that lose sensor–skin coupling.Contact favored the radial side of the index-pinky fingers and the ulnar side of the thumb rather than the centerline.Source: A. Tactile Sensing Design · p. 4, Figure 4 · p. 4
  • 65% fingertip coverage across a broad population is maintained in one glove size using approximately 7 N/cm sensors and footprints under 50% of finger circumference.Effectively inextensible sensors limit coverage to approximately 40% or require additional sizes.Source: A. Tactile Sensing Design · p. 4
  • 47 sensor locations use dense 4×2 arrays on distal phalanges and at most single-element sensors elsewhere.A 12 pF reference capacitor improves ADC resolution and reduces minimum detectable force.Source: A. Tactile Sensing Design · p. 4

B. Pose Tracking Design

  • Three 6-axis IMUs on each digit and one dorsum root reference fuse with egocentric vision, which supplies global position while IMUs update joint angles through occlusion.The 16 IMUs omit magnetometers to avoid failure near ferrous objects or active magnetic fields.Source: B. Pose Tracking Design · p. 4

C. Supporting Electronics

Compliant routing and textile-mounted electronics are designed to avoid restricting finger motion or locally stiffening the glove.

C. Supporting Electronics

  • Coiled conductors achieved ≥70% elongation at 0.60 N, >25 N failure load, and 168K cycles to 70% elongation.The mylar-wrapped, kevlar-core construction with TPU overmold provides stretch at a spring rate low enough to avoid perceptible resistance during natural finger motion.Source: C. Supporting Electronics · p. 4
  • Dorsum boards sit in knit textile pockets rather than being overmolded, preserving textile elasticity and avoiding localized stiffening that would increase encumbrance.Source: C. Supporting Electronics · p. 5
  • Each finger carries three phalange-level boards with a microcontroller, IMU, and analog sensor connections, linked over a UART finger bus.A dorsum controller stack provides wireless communication, storage, and USB-C; the 900 mAh battery supports 3.5+ hours of untethered operation.Source: C. Supporting Electronics · p. 5

IV. SIGNAL MODELING

Tactus combines pneumatic factory characterization with per-session online scaling to calibrate tactile force readings. The latter improves force accuracy, while observed global drift motivates recalibration across sessions.

A. Tactile Calibration Pipeline

  • The factory rig characterizes all 47 sensor locations simultaneously in under 23 min using repeatable pressure stimuli and per-sensor cubic transfer functions.It extracts full span, sensitivity, hysteresis, noise, baseline, and fitting error.Source: A. Tactile Calibration Pipeline · p. 5
  • Over 15 hours of forceful use across three gloves showed no localized sensor degradation, but full span fell about 3% per don/doff session and hysteresis grew about 3% per session on test gloves.Zero-load drift on two gloves was mitigated by zero-offset taring; the authors state per-session calibration is needed for drift and don/doff, fit, and grounding differences.Source: A. Tactile Calibration Pipeline · p. 5
  • Online calibration compares summed fingertip force, calculated from sensor area and factory-calibrated pressure, with a reference force to fit a scaling factor.The fitted factor corrects the factory calibration using a linear predicted-versus-actual force fit.Source: A. Tactile Calibration Pipeline · p. 5
  • A single online scaling factor improves median force RMSE by 1.7–2.9× and median relative error by 1.5–2.4× over factory calibration alone.The authors attribute the dominant residual to wearer-specific gain from fit, grounding, and hand pose.Source: A. Tactile Calibration Pipeline · p. 5, Table I · p. 5

B. Hand Pose Tracking Pipeline

The pose pipeline offers real-time vision–IMU fusion and an offline full-sequence solver for different tracking needs. Its evaluation compares tracking modes with marker-based mocap ground truth during manipulation tasks.

V. PERFORMANCE EVALUATION

  • Tracking modes are evaluated against OptiTrack marker-based mocap across 16 manipulation tasks performed by five participants; MKPE and fingertip-restricted F.MKPE measure position error.Translation-aligned variants are also reported.Source: B. Hand Pose Tracking Pipeline · p. 5

B. Hand Pose Tracking Pipeline

  • Real-time CV+IMU EKF fusion asynchronously combines visual keypoints with IMU measurements, while AVI-HT fuses video and IMU tokens through cross-sensor attention.The EKF propagates state and covariance, then corrects estimates against the observations.Source: B. Hand Pose Tracking Pipeline · p. 5
  • The offline AVI-HT full-sequence solver refines trajectories using complete temporal context, trading real-time capability for accuracy in dataset curation.Source: B. Hand Pose Tracking Pipeline · p. 5

V. PERFORMANCE EVALUATION

  • The evaluation protocols measure natural-hand task interference, tactile-force sensing quality, and hand-pose estimation quality.Source: §V · p. 6

A. Protocol Dexterous-Tasks

Across dexterous-task, tactile, and hand-pose protocols, Tactus is evaluated for preserving natural manipulation and providing useful sensing. The benchmarks compare tactile force regression and pose tracking with baselines, while noting that tactile calibration results from different protocols are not directly comparable.

A. Protocol Dexterous-Tasks

  • Tactus had the best performance among tested wearables across three encumbrance metrics, with almost barehand task-completion time and a slight subjective-workload trade-off.The within-participant study included five participants and 23 household, personal, and standardized dexterity tasks.Source: A. Protocol Dexterous-Tasks · p. 6

B. Protocol Tactile

  • Tactus achieved the lowest minimum detectable force, highest cross-participant consistency, and lowest pose-related artifact ratio versus PPS and JQ gloves.JQ had slightly better point accuracy: 0.18 N versus Tactus at 0.20 N; neither commercial glove had a documented calibration pipeline for maintaining performance across subjects or sessions.Source: B. Protocol Tactile · p. 6, Table III · p. 6
  • The tactile benchmark’s learned multi-channel MLP estimates fingertip normal force at lighter forces, unlike the per-sensor cubic transfer functions evaluated across all 47 locations and the full pressure range.These results are not directly comparable across the two calibration evaluations.Source: B. Protocol Tactile · p. 6

C. Protocol Hand Pose

  • All fused tracking modes outperformed the CV-barehand baseline; adding IMUs reduced wrist-aligned error by 11% and fingertip error by 17%, but absolute error improved by under 3%.AVI-HT showed its largest gains on absolute metrics, while full-sequence optimization further improved all four metrics; the supplied passage does not give those final values.Source: C. Protocol Hand Pose · p. 6

VI. APPLICATIONS: FORCE-DEPENDENT TASKS

The applications section examines force-dependent tasks supported by Tactus signals. For surface material estimation, it compares tactile classification with a vision-language model on nine roughness categories.

VI. APPLICATIONS: FORCE-DEPENDENT TASKS

  • The applications evaluate force-dependent tasks as capabilities supported by Tactus tactile and pose data.The supplied passage names object material classification, slip detection, and multisensory hand-object manipulation.Source: §VI · p. 7

A. Surface Material Estimation

  • For nine-category roughness classification, participants slide an index finger across texture pads while a tactile network is compared with a zero-shot vision-language model.The tactile model uses transformer modules and an MLP head; the vision-language baseline predicts category from an egocentric image and text prompt.Source: A. Surface Material Estimation · p. 7

B. Slip Detection

Slip detection is tested on repeated stick-slip events, and object manipulation modeling evaluates fused glove and vision signals under induced occlusion. The reported slip result is a feasibility finding, with generality still under investigation.

B. Slip Detection

  • The slip rig generates repeated natural stick-slip cycles, with NO-SLIP, PRE-SLIP, and SLIP labels derived from encoder position and tangential load.A spring ramps the pull force until static friction breaks, then decompresses so the sled re-sticks.Source: B. Slip Detection · p. 7
  • A causal temporal CNN detected slip with F1 = 1.00 and fired within 0.01 sec of labelled onset on one train/test split.Evaluation covered seven sessions, with five training and two testing sessions and about 34 slip events each; Tactus measured pressure without force decomposition.Source: B. Slip Detection · p. 7
  • The slip account remains under-determined because a normal-pressure distribution does not uniquely specify shear, so the result is treated as feasibility only.Planned follow-up covers additional preloads, slip directions, and a second held-out group.Source: B. Slip Detection · p. 7

C. Object Manipulation Modeling

  • Under induced hand-object and hand-hand occlusion, joint optimization of Tactus pose and tactile information can improve tracking, while contact constrains hand-object kinematics.The comparison with HaMeR and FoundationPose is qualitative and uses pose overlays on the original video.Source: C. Object Manipulation Modeling · p. 7

VII. CONCLUSION AND FUTURE WORK

Tactus combines calibrated fingertip force sensing and occlusion-resilient hand tracking in a textile form factor intended to preserve natural grasp behavior. The paper presents benchmark performance and identifies larger-scale evaluation and reduced calibration burden as future work.

VII. CONCLUSION AND FUTURE WORK

  • The system combines textile-integrated calibrated fingertip force sensing with occlusion-resilient hand tracking and a form factor that preserves natural grasp behavior.Its measurement-system contribution includes empirically informed sensor placement, a documented calibration pipeline with quantified drift, and standardized benchmarking protocols.Source: §VII · pp. 7–8
  • Tactus benchmarks report 0.59 N minimum detectable force at 95% confidence, 11.0 mm root-aligned fingertip error after offline optimization, and task completion time at 106% of barehand.The protocols specify ground truth, metrics, and drift reporting to support reproducible quantitative comparison across sensing gloves.Source: §VII · p. 8
  • Future work includes larger studies across participants, surfaces, slip detection, and glove platforms, reducing per-session calibration burden, and reporting full-life-cycle performance degradation.The protocols are specified to allow these extensions by the authors and others.Source: §VII · p. 8

Related papers

Loading 2610.39076v1…