PaperlayerView exact revision on arXiv

Source-linked AI summary

CNN-based Methods for Object Recognition with High-Resolution Tactile Sensors

Juan M. Gandarias, Alfonso J. García-Cerezo, Jesús M. Gómez-de-Gabriel

arXiv:2305.12417v1cs.CVcs.AIcs.RO

TL;DR

Tactile object recognition must interpret pressure images, but large RGB-trained CNNs may be inefficient for lower-resolution tactile inputs and resource-constrained robots. The paper compares transfer learning and custom tactile-trained CNNs across 11 configurations. CNN methods achieve high accuracy, with TactNet offering similar recognition rates and substantially shorter classification times than transfer learning.

  • Problem

    Large RGB-trained CNNs may be inefficient for tactile images with fewer features and lower resolution, particularly in embedded applications with computational and energy constraints.

    Source: §I · p. 2

  • Method

    The paper compares 8 transfer-learning configurations using RGB-pretrained CNNs with 3 TactNet configurations trained from scratch on tactile data.

    Source: §I · p. 2

  • Results

    CNN approaches achieve high accuracy; ResNet-NN improves mean recognition accuracy by up to 17.72% against SURF-SVM, while TactNet has significantly shorter classification times.

    Source: §VI · p. 8

  • Takeaways & Limitations

    TactNet achieves similar recognition rates with a huge reduction in classification time, and full from-scratch training is described as a worthwhile one-time operation.

    Source: §VII · p. 9

Abstract

from arXiv · show

Novel high-resolution pressure-sensor arrays allow treating pressure readings as standard images. Computer vision algorithms and methods such as Convolutional Neural Networks (CNN) can be used to identify contact objects. In this paper, a high-resolution tactile sensor has been attached to a robotic end-effector to identify contacted objects. Two CNN-based approaches have been employed to classify pressure images. These methods include a transfer learning approach using a pre-trained CNN on an RGB-images dataset and a custom-made CNN (TactNet) trained from scratch with tactile information. The transfer learning approach can be carried out by retraining the classification layers of the network or replacing these layers with an SVM. Overall, 11 configurations based on these methods have been tested: 8 transfer learning-based, and 3 TactNet-based. Moreover, a study of the performance of the methods and a comparative discussion with the current state-of-the-art on tactile object recognition is presented.

I. INTRODUCTION

Tactile perception requires both sensing and the ability to interpret tactile information, while CNNs developed for RGB images may be inefficient for constrained robotic applications. This paper compares transfer learning and custom tactile-trained CNNs for pressure-image recognition.

I. INTRODUCTION

  • Piezoresistive tactile sensors represent measured tactel pressure as image pixels, making pressure arrays amenable to image-based recognition methods.The introduction also notes that sensor technologies differ in the physical quantities they measure.Source: §I · p. 1
  • For embedded applications, large pre-trained CNNs may be inefficient because tactile images have fewer features and lower resolution than RGB images, while computational and energy constraints also matter.The introduction notes that custom CNNs may better fit smaller inputs or fewer output labels, with data augmentation addressing limited training data.Source: §I · p. 2
  • The paper compares 11 CNN-based approaches for pressure-image recognition, including transfer learning from RGB-trained CNNs and custom CNNs trained from scratch with tactile information.The evaluation includes a 22-class experiment and compares recognition accuracy and classification time.Source: §I · p. 2
  • The tactile dataset and methods are made available for comparison with new approaches and reproducibility.The supplementary material includes the dataset and Matlab code.Source: §I · p. 3

II. RELATED WORK

Related work distinguishes material or surface identification from object recognition and describes tactile-recognition methods using AI, image features, sequences, and multimodal sensing.

II. RELATED WORK

  • Material identification targets surface properties such as roughness, texture, stiffness, or thermal conductivity, for which sequential tactile data can be important.One cited approach uses exploratory arm motion and a multi-channel neural network with a 2D force sensor.Source: §II · p. 3
  • Object-recognition work distinguishes objects by shape, and related neural-network methods exploit CNN translation invariance to recognize contacts across image positions.Examples include recognizing either hand with artificial skin and combining kinesthetic and tactile information for shape recognition.Source: §II · p. 3
  • Multimodal tactile methods combine sensing modalities; cited work reports that haptic and visual fusion generally performs better than a single information source.A cited CNN method uses accelerometer information and a surface-material image to estimate contact material.Source: §II · p. 3
  • Image-based tactile recognition commonly extracts pressure-image features before classification, but a cited SIFT-variant and kNN solution needs multiple touches for good results.For food-texture classification, a CNN processes pressure-image sequences, although two images produced results similar to using a sequence.Source: §II · p. 3

III. METHODOLOGIES

The methods treat tactile pressure as images and compare transfer learning from RGB-trained CNNs with custom networks trained on tactile data. Transfer learning requires resizing pressure images to fit the pre-trained network and permits either an SVM or fine-tuned classification layers.

A. Tactile images

  • Flexible sensors bending around an object produce higher contact area but lower dynamic range than a flat rigid sensor; the rigid configuration has higher pressure values and was selected for the experiments.The chosen arrangement includes a silicon-rubber layer to protect the film and enhance sensor depth of field.Source: A. Tactile images · p. 3, Figure 3 · p. 4

B. Transfer Learning

  • Transfer learning reuses convolutional layers from an RGB-trained CNN to extract features from pressure images, while replacing its classification layers with a custom classifier.The paper considers pre-trained feature extractors and describes convolutional layers as feature extractors and fully connected layers as classifiers.Source: B. Transfer Learning · p. 3
  • Because the pre-trained network requires its original input dimensions and three channels, tactile images are resized and copied across three channels.The resizing uses bicubic interpolation, computing each resized pixel from 16 neighboring pixels.Source: B. Transfer Learning · pp. 4–5
  • The transfer-learning classifier is either an SVM replacing the classification layers or those layers fine-tuned on tactile data.These are the two alternatives shown for the transfer-learning approach.Source: Figure 4 · p. 4, B. Transfer Learning · p. 5

C. TactNet

TactNet is a custom CNN trained from scratch using tactile information. The paper considers plain-layer and residual-layer configurations for learning tactile features and classifying pressure images.

C. TactNet

  • TactNet is a custom CNN trained from scratch on pressure-image data, although this approach requires a large amount of tactile data.Source: C. TactNet · p. 5
  • The three configurations use either four or six plain layers following AlexNet, or six layers with residual convolutions following ResNet.Source: C. TactNet · p. 5
  • Convolutional or residual layers extract tactile features, while batch normalization, ReLU, optional max pooling, and fully connected layers support classification.Batch normalization follows each convolution with ϵ = 10^-4; some layers are alternated with stride-2 max pooling.Source: C. TactNet · p. 5

1) TactNet-4:

TactNet-4 is the simplest TactNet configuration, using three convolutional layers and a 22-neuron fully connected classifier followed by softmax.

1) TactNet-4:

  • TactNet-4 uses three convolutional layers with filter sizes 5 × 5, 3 × 3, and 3 × 3 and 8, 16, and 32 filters, respectively.Source: 1) TactNet-4 · p. 5
  • A 22-neuron fully connected layer followed by softmax produces class likelihoods for the input tactile data.Source: 1) TactNet-4 · p. 5
  • The network structure is shown as panel a) in Fig. 5.Source: 1) TactNet-4 · p. 5

2) TactNet-6:

TactNet-6 uses five convolutional layers followed by a 22-neuron classifier and softmax; its structure is shown in Fig. 5b.

2) TactNet-6:

  • TactNet-6 has five convolutional layers with 8, 16, 32, 64, and 128 filters of respective sizes 5 × 5, 5 × 5, 3 × 3, 3 × 3, and 3 × 3, followed by a 22-neuron fully connected layer and softmax.Source: 2) TactNet-6 · p. 5
  • A 22-neuron fully connected layer followed by softmax produces the class probabilities.Source: 2) TactNet-6 · p. 5
  • Fig. 5 identifies TactNet-6 as configuration b).Source: Figure 5 · p. 5, 2) TactNet-6 · p. 5

3) TactResNet:

TactResNet combines a convolutional layer, four residual layers, and a fully connected layer followed by softmax. The experiment description covers 22 object classes and compares 11 CNN-based procedures.

3) TactResNet:

  • TactResNet comprises one convolutional layer, four residual layers, and one fully connected layer followed by softmax.Source: 3) TactResNet · p. 5

A. Description of the experiments

B. Sensor specifications

The sensor is mounted on a robotic manipulator and protected by a silicone-rubber contact interface. Its pressure array contains 1,400 tactels arranged in a 28-by-50 matrix.

B. Sensor specifications

  • A 3D-printed PLA component couples the sensor to the AUBO Our-i5 six-degree-of-freedom robotic arm.The part was made using fused deposition modelling.Source: B. Sensor specifications · p. 6
  • The Tekscan model 6077 array contains 1,400 pressure sensels arranged in 28 rows by 50 columns.The sensor has a density of 27.6 tactels/cm2.Source: B. Sensor specifications · p. 6
  • A silicone-rubber pad provides soft contact with objects, while the sensor covering protects the sensor and conducts external forces.Source: B. Sensor specifications · p. 6

C. Dataset

The study uses ImageNet-pretrained CNNs and a tactile dataset to train and evaluate transfer-learning and TactNet methods. Across randomized samples, TactNet configurations achieve high accuracy, and the section also describes comparisons across methods and sensor resolutions.

C. Dataset

  • The tactile dataset contains 1,100 pressure images across 22 labeled classes, divided into 704 training, 176 validation, and 220 test images.The transfer-learning CNNs were pretrained on more than a million ImageNet images spanning 1,000 categories.Source: C. Dataset · p. 6
  • TactNet training uses reflections, rotations, and translations, expanding the training data to 4,224 images.The augmented validation and test sets contain 1,056 and 1,320 images, respectively.Source: C. Dataset · p. 6

V. RESULTS

  • Each TactNet configuration’s learning-process accuracy was close to 100%.Fig. 6 shows the learning processes from scratch.Source: Figure 6 · p. 7, §V · p. 7
  • Mean recognition rates exceeded 93% for all TactNet configurations across 20 randomized samples.The samples were split into training, validation, and test sets; Fig. 7 reports means and one-standard-deviation error bars.Source: Figure 7 · p. 7, §V · p. 7
  • The methods’ mean recognition rates were compared over 20 randomized samples, alongside parameter counts and GPU classification times.Fig. 8 compares recognition rates, while Fig. 9 compares parameter counts and classification time on an NVIDIA GeForce GTX 1050Ti GPU.Source: Figure 8 · p. 7, §V · p. 7, Figure 9 · p. 7
  • The best method from each approach was evaluated at approximately 1/16, 1/8, 1/4, 1/2, and full sensor resolution using one touch.Fig. 10 presents the comparison; resolution was reduced by bicubic interpolation, and TactNet6’s structure and some hyperparameters were adjusted to the input dimensions.Source: §V · pp. 7–8, Figure 10 · p. 8

VI. ANALYSIS AND DISCUSSION

CNN methods outperform SURF-SVM on recognition accuracy in the reported comparison, while TactNet offers similar recognition rates to the best transfer-learning methods with shorter classification times. The paper also compares its results with prior tactile-recognition work and reports further time reduction at lower resolutions.

VI. ANALYSIS AND DISCUSSION

  • TactNet configurations achieve recognition rates similar to the best transfer-learning methods with substantially shorter classification times.The paper reports that transfer-learning time increases when larger CNNs are used and their fully connected layers are fine-tuned.Source: §VI · p. 8
  • ResNet-NN improves mean recognition accuracy by up to 17.72% against SURF-SVM, while SqueezeNet-SVM improves it by 8.08%.Table IV summarizes the method comparison.Source: Table IV · p. 8, §VI · p. 8
  • Lower sensor resolutions provide a further reduction in TactNet computation time.Source: §VI · p. 8
  • Among compared tactile-image studies with more than two classes, the paper reports higher accuracy than all except, which classifies 25 symbols.The authors note that their methods use only one touch and outperform methods with an order of magnitude fewer classes.Source: §VI · p. 8

VII. CONCLUSION

The paper presents and compares 11 CNN-based methods for recognizing 22 object and human-body-part classes from one tactile touch. Its conclusion highlights similar recognition rates with substantially shorter classification times for TactNet, and identifies temporal tactile sequences as future work.

VII. CONCLUSION

  • The authors report that CNN methods, including SURF-SVM comparisons, obtain higher recognition accuracy than other recent tactile-image approaches in their comparison.The state-of-the-art table includes studies treating tactile data as common images; the last three entries are the paper’s methods.Source: §VII · p. 9, Table V · p. 9
  • The 11-method comparison finds that TactNet can achieve recognition rates similar to the best transfer-learning methods with a substantial reduction in classification time.The methods were evaluated on the same tactile image dataset and hardware.Source: §VII · pp. 8–9
  • Future work will study sequences of tactile images to use contact dynamics for object classification and assess real-time 3D CNN performance and computational requirements.Source: §VII · p. 9

Related papers

Loading 2305.12417v1…