PaperlayerView exact revision on arXiv

Source-linked AI summary

SpectraTac: A Compact Camera-Free Optical Tactile Sensor with Distributed Color Sensing

Hao Wu, Haotian Guo, Yu Feng, Yutong Wang, Yanzhe Wang, Jianshu Zhou

arXiv:2608.30368v1cs.RO

TL;DR

Combining rich tactile information with compact hardware, low cost, and low computational requirements remains challenging. SpectraTac addresses this gap with a compact, camera-free sensor using active RGB illumination and distributed color sensing, achieving accurate force estimation and contact-region classification without image processing.

  • Problem

    Combining rich tactile information with mechanical compliance, compact packaging, low computational requirements, and straightforward integration remains challenging, while color-resolved signals can increase tactile information density without substantially increasing sensor size.

  • Method

    SpectraTac combines active RGB illumination with three spatially distributed RGBC sensors to encode deformation-induced changes in internal light propagation as low-dimensional spatio-spectral signals for multitask decoding.

  • Results

    The model achieved R2 > 0.91 for all six mechanical components, force MAEs below 0.2 N for in-plane shear and below 0.5 N for normal force, and 99.9% contact-region classification accuracy.

  • Takeaways & Limitations

    Distributed optoelectronic sensing supports accurate 3D force estimation, contact-region classification, real-time force tracking, and contact-region-based human–machine interaction without image acquisition.

Abstract

from arXiv · show

Tactile sensing is essential for physical interaction in robotics and human--machine systems. However, combining rich tactile information with compact hardware, low cost, and low computational overhead remains challenging. This work presents SpectraTac, a compact, camera-free optical tactile sensor that combines active red--green--blue (RGB) illumination with spatially distributed color sensing. Contact deforms a compliant transparent elastomer and modulates its internal light field, producing spatially differentiated changes in color and intensity. Three distributed color sensors capture these responses as low-dimensional spatio-spectral features, avoiding cameras, imaging optics, and high-dimensional image processing. The device measures 19.2 mm in diameter and 4 mm in height, with a material cost below USD~5. A data-driven decoding framework simultaneously estimates three-dimensional (3D) force and the contact region from the optical measurements. For 3D force prediction, the sensor achieved mean absolute errors (MAEs) of 0.161, 0.164, and 0.429 N along the x-, y-, and z-axes, respectively. The nine-region contact-classification accuracy was 99.9%. We further evaluated real-time 3D force tracking and contact-region-based human--machine interaction through an interactive control task. These results indicate that distributed color-resolved optical sensing offers a compact, low-cost alternative to camera-based tactile sensing for robotics, wearable sensing, and interactive systems.

I. INTRODUCTION

SpectraTac addresses the challenge of combining rich tactile information with compact, low-cost, low-computation hardware by using camera-free, color-resolved optical sensing. Its distributed RGB illumination and color sensors support force and contact-region estimation without image acquisition.

  • Motivation: Tactile sensing provides contact, force, deformation, surface, and slip information that vision alone cannot always infer reliably.These capabilities support grasp stability and adaptability in robotic grippers and dexterous hands.
  • Motivation: Color-resolved optical signals provide multiple sensing channels for the same deformation while maintaining compactness and low computational complexity.Prior work used wavelength-dependent attenuation, multiwavelength referencing, and chromaticity changes for multipoint bending, pressure, and positional sensing.
  • Contribution: SpectraTac combines active RGB illumination with three spatially distributed color sensors in a compact, camera-free optical tactile architecture.Contact-induced deformation modulates the internal light field of a compliant transparent PDMS layer, generating spatially differentiated color and intensity changes.
  • Contribution: The low-dimensional spatio-spectral representation is decoded to estimate 3D force and contact region without image acquisition or high-dimensional image processing.The platform integrates the compliant medium, illumination, and color-detection electronics for robotics, wearable sensing, and human–machine interfaces.

II. SENSOR DESIGN

The sensor integrates compliant optical material, illumination, and distributed color detection into a fingertip-scale cylindrical package. Alternating emitters and receivers provide multidirectional sensing of contact-induced deformation.

  • Structural Design: 19.2 mm in diameter and 4 mm in height, the sensor integrates a transparent PDMS layer, shielding layer, three RGBC sensors, and three RGB illumination units.The compact structure facilitates integration into robotic platforms and human–machine interfaces.
  • Structural Design: Three RGBC sensors and three RGB illumination units are arranged alternately around the circumference, with same-type components spaced at 120° intervals.Adjacent alternating units are separated by 60°.
  • Structural Design: The distributed configuration senses contact-induced deformation from multiple directions and generates spatially differentiated optical responses.The transparent PDMS layer serves as both the compliant sensing medium and optical propagation medium.

B. Working Principle

Contact deforms the transparent elastomer and changes internal light propagation, producing spatially distinct RGBC responses. Their combined spectral and spatial information forms a compact representation for decoding force and contact position.

  • Working Principle: Contact changes local interface geometry, incidence angles, and optical path lengths, redistributing the internal light field through reflection, scattering, attenuation, and leakage.The three distributed RGBC sensors therefore respond differently to contact locations and loading conditions.
  • Working Principle: Each RGBC sensor measures red, green, blue, and clear-channel intensities, and combining three sensors yields a compact spatio-spectral response vector.The channel intensities are denoted Ri, Gi, Bi, and Ci for sensor i.
  • Working Principle: Spectral intensity variations encode deformation magnitude, while differences among the three sensors capture spatial asymmetry caused by loading.These complementary responses support estimation of force magnitude, loading direction, and contact position.
  • Fabrication: Commercial components and a simple molding process produced the assembled sensor at a material cost below USD 5 per sensor.PDMS provides optical transparency and mechanical compliance, while the opaque shielding layer suppresses ambient light and unwanted external reflections.

III. TACTILE SIGNAL REPRESENTATION AND DECODING

Calibration pairs distributed RGBC measurements with six-axis force/torque targets and discrete contact-region labels. The resulting dataset supports simultaneous continuous mechanical-state regression and nine-region contact classification.

  • Ground-Truth Acquisition and Calibration: A six-axis force/torque sensor provides synchronized reference measurements for mapping optical outputs to tactile states during controlled contacts.The reference state contains three orthogonal force components and three corresponding torque components.
  • Ground-Truth Acquisition and Calibration: Each contact sample receives a discrete label for one of nine representative sensing-surface regions, enabling simultaneous contact-region classification.Samples without reliable region annotations are handled separately during training.
  • Dataset Construction: Each training sample comprises an optical measurement vector, a continuous force/torque target, and a discrete contact-region label.The optical measurement vector is described as x(n) ∈ R12, while the label set includes −1 for unavailable region annotations.
  • Dataset Construction: Approximately 60,000 samples were collected over 50 minutes at 20 Hz, providing high acquisition throughput and short calibration time.Labels −1 were retained for force/torque regression but excluded from the contact-region classification loss.

B. Physically Motivated Feature Representation and Normalization

SpectraTac expands distributed optical measurements into a 33-dimensional representation that combines chromaticity ratios with pairwise sensor differences before normalization. Zero-anchored target scaling preserves the physical zero-force and zero-torque reference.

  • Feature construction: The sensor produces a 12-dimensional raw optical measurement vector that is expanded into a 33-dimensional feature representation.Feature expansion is applied directly to raw optical intensities before statistical normalization.
  • Feature construction: Chromaticity ratios decouple spectral shifts from broadband intensity attenuation across the three distributed color sensors.A small regularization constant prevents division by zero under low-light conditions.
  • Feature construction: Pairwise sensor differences represent optical-flux asymmetry induced by tangential shear forces and multiaxis moments.The differences are computed for sensor pairs (1,2), (2,3), and (3,1).
  • Normalization: The complete feature vector concatenates the physically motivated chromaticity and pairwise-difference groups before z-score normalization.The normalized network input uses training-sample means and standard deviations.
  • Normalization: Zero-anchored target scaling preserves zero force and zero torque at zero in normalized target space by omitting mean subtraction.The standard deviation for each F/T component is computed exclusively from the training set.

C. Shared-Backbone Multitask Learning and Masked Optimization

A shared-backbone residual multilayer perceptron processes the engineered optical features and branches into force/torque regression and contact-region classification. Masked multitask optimization retains samples without region labels for regression while excluding them from classification loss.

  • Model architecture: A shared-backbone residual multilayer perceptron branches into six-axis F/T regression and nine-class contact-region classification.The classification branch converts logits into probabilities and selects the predicted region with argmax.
  • Optimization: The masked multitask objective combines F/T regression and contact-region classification losses under a One-Cycle learning-rate schedule.Training-loss curves are shown in Fig. 3(c).
  • Optimization: The F/T regression head uses a component-weighted L1 loss that assigns larger relative weights to in-plane forces and smaller weights to torque components after target scaling.The weights correspond to [Fx, Fy, Fz, Mx, My, Mz] and do not represent unit conversions.
  • Optimization: Samples with unavailable region annotations are excluded from classification loss but retained for continuous F/T regression.The label value −1 identifies samples without reliable contact-region annotations.

D. Decoding Performance

SpectraTac simultaneously estimates continuous mechanical quantities and discrete contact regions with strong held-out performance. Regression accuracy was high across all six components, while contact-region classification reached near-perfect accuracy.

  • Regression: R2 > 0.91 was achieved for all six mechanical components on the unseen test set.The six components include three forces and three rotational axes.
  • Regression: MAEs were below 0.2 N for in-plane shear forces and below 0.5 N for normal force.The corresponding absolute-error distributions are reported for the 3D forces.
  • Regression: MAE and RMSE were at most 0.004 N·m for all three rotational axes.Figure 3(g) summarizes the overall regression performance.
  • Classification: 99.9% contact-region classification accuracy was achieved with the masked classification head.The remaining errors occurred between adjacent regions.

IV. APPLICATION AND DEMONSTRATION

The sensor was evaluated in real-time force estimation and tactile human–machine interaction. Both demonstrations used the same compact optical architecture to provide continuous force and discrete contact-region information.

  • Applications: Real-time 3D force estimation was assessed under continuously varying contact conditions.The second application used contact-region classification for human–machine interaction through a tactile game interface.
  • Applications: The interactive demonstration used contact-region classification for human–machine interaction through a tactile game interface.Both applications relied on the same compact optical architecture.

A. Real-Time Three-Dimensional Force Estimation

SpectraTac enabled real-time 3D force estimation from distributed RGBC measurements during continuously varying loads. Its predictions closely tracked simultaneously acquired reference force/torque readings across dynamic loading profiles.

  • A. Real-Time Three-Dimensional Force Estimation: Real-time 3D force estimation used distributed RGBC measurements processed by a trained model.The estimated force was represented as ˆF = [ˆFx, ˆFy, ˆFz]T.
  • A. Real-Time Three-Dimensional Force Estimation: The predicted forces closely tracked reference F/T signals during continuously varying dynamic loads.Evaluation included repeated short contacts and other dynamic loading profiles shown in Fig. 4(b) and (c).

B. Contact-Region-Based Human–Machine Interaction

SpectraTac supported contact-region-based human–machine interaction by converting compliant-surface optical responses into directional commands for a real-time Snake game. The demonstration combined continuous physical-state estimation with discrete commands while preserving mechanical flexibility.

  • B. Contact-Region-Based Human–Machine Interaction: Nine sensing regions were mapped to directional commands, with the center as neutral and peripheral regions grouped into four directions.Commands were executed only when normal force exceeded 1.0 N to avoid accidental triggers.
  • B. Contact-Region-Based Human–Machine Interaction: A classification head continuously predicted contact regions from distributed optical responses while a user pressed different peripheral areas.Predictions in designated zones steered the Snake in real time.
  • B. Contact-Region-Based Human–Machine Interaction: The interface relied on deformation-induced optical responses within a compliant sensing surface rather than conventional mechanical buttons or touchscreens.This preserved the sensing surface’s mechanical flexibility.
  • B. Contact-Region-Based Human–Machine Interaction: The same low-dimensional optical signals supported continuous physical-state estimation and discrete interaction commands.The demonstration suggests potential applications in robotic control, handheld interfaces, and human–machine interaction systems.
  • B. Contact-Region-Based Human–Machine Interaction: The conclusion identifies distributed optoelectronic sensing as a compact alternative to camera-based tactile sensors for robotics and human–machine interaction.Future work includes improving spatial resolution, force decoupling, dynamic tactile perception, and long-term robustness.
Loading 2608.30368v1…