VibroTact: Soft Piezo Vibration Fingertip Sensor for Recognition of Texture Roughness via Robotic Sliding Exploratory Procedures

This letter presents a low-cost soft fingertip sensor for recognition of texture roughness. This sensor is designed with a soft rounded shape made of silicone rubber. An array of flexible piezo vibration elements, embedded in the sensor, measures vibrations while interacting with an object surface. The sensor is mounted on a robot arm for data collection with sliding exploratory procedures. The soft fingertip sensor is validated with roughness recognition of eight artificial textures using five different sliding directions (horizontal, vertical, left diagonal, right diagonal, and square). The sensor uses four machine learning methods [K-nearest neighbors (KNN), support vector machines (SVM), artificial neural networks (ANN), and convolutional neural networks) for recognition and comparison processes. The tactile sensor can recognize textures with accuracies from 90.60% to 100% for different sliding directions and computational methods, where the horizontal sliding with SVM and KNN methods achieved the highest performance. This tactile sensor has the potential to extract surface properties for autonomous object exploration, recognition, and manipulation tasks.

magnetometer elements) sliding at constant speed [20].A rigid tip with six-axis force/torque sensor recognized 21 textures with sliding EP in three directions with 98.8% accuracy [21].
In this work, the VibroTact, a soft fingertip with vibration sensing elements is presented for perception of texture roughness using sliding EPs (see Fig. 1).The proposed fingertip sensor is designed with a soft rounded shape body and an embedded array of flexible piezo vibration sensors.The VibroTact is mounted on a robot arm for the exploration of eight objects with ridged surfaces.The capability for the recognition of surface roughness is validated by sliding the sensor on each ridged surface with five different sliding directions (horizontal, vertical, left diagonal, right diagonal, and square) to identify their effect in perception accuracy.The data are processed and analyzed with a set of computational methods.The experiments show that the VibroTact sensor can recognize surface roughness with a range of accuracy performance according to the direction of the sliding EP.
The rest of this letter is organized as follows.The sensor design and experimental setup are in Section II.The experiments and results are in Section III.Finally, Section IV concludes this article.

A. VibroTact Design
Soft fingertip design: The VibroTact sensor is designed with a rounded shape geometry, inspired by previous works [14], [15], and made of Ecoflex 00-30 soft rubber material from Bentley, Inc. [22], [23].The soft fingertip has dimensions of 40 mm diameter and 20 mm height, as shown in Fig. 2. The soft body of the sensor has four cavities or slots symmetrically located to embed four piezo vibration elements.The VibroTact sensor is attached to a 3-D printed holder to be mounted on the flange of a robotic arm for the implementation of sliding EPs.The soft structure of the rounded fingertip sensor allows the piezo material elements to vibrate while being in contact and sliding on the object surfaces.Fig. 2 shows the CAD of the sensor design and its different components.
Vibration sensing elements: The soft body of the VibroTact sensor is embedded with four flexible piezo vibration elements LDT0-028K, from TE Connectivity MEAS.This sensing element comprises a 28-µm thick piezo polymer film and laminated to a polyester layer.The piezo vibration element has dimensions of 25 mm × 13 mm × 0.205 mm for length, width, and depth, respectively.When the flexible sensor is hold from the contact terminals and left to vibrate, due to the interaction with an object surface, voltage outputs are generated.The four flexible sensors (piezo 1, piezo 2, piezo 3, and piezo 4) are placed symmetrically in the fingertip (see Fig. 2).These piezo element locations allow the detection of vibrations while the sensor slides along different directions on object surfaces.The contact terminals of each piezo element are connected to the conditioning circuity and then to an Arduino UNO microcontroller for data collection.

B. Robot Platform and Artificial Surface Textures
The VibroTact sensor is mounted on the six-axis Niryo One robot arm to slide the sensor on artificial textures for systematic data collection.Fig. 3(a) shows the robot arm with the tactile sensor.Eight 3-D printed ridged surfaces or artificial textures (labels T1-T8) with different roughness are used for the perception tasks.The roughness of these textures vary in terms of 1) the distance or gap between

C. Data Acquisition Interface
An interface composed of Arduino boards, the Niryo One software, and LabView has been developed for systematic data collection.The components of this interface are shown in Fig. 3(d).The robot is programmed to move to a predefined initial position.Then, the sensor starts the sliding EP in a specific direction, e.g., horizontal direction, on the artificial texture.The VibroTact is slid on the texture for a predefined number of repetitions.The tip of the sensor is covered with a thin film to avoid damaging the soft material without affecting the data measurement.The sensor data are measured and conditioned with a voltage divider circuitry, with voltage values ranging from 0 to 1.1 V, to be read by the Arduino boards.A program and graphical user interface developed in LabView allows the monitoring, visualization, and automated recording of data from the array of four piezo vibration elements for subsequent analysis.

A. Sensor Data Collection
The VibroTact sensor is placed on top of eight artificial textures (T1-T8) for data collection.Each texture has a specific roughness defined by bumps of different diameter and distance between them (see Section II-B).Fig. 4(a) shows the textures explored by sliding the tactile sensor in five different directions (horizontal, vertical, left Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.diagonal, right diagonal, and square).For each texture, the sensor is slid at a constant speed of 15 mm/s collecting 900 data samples.This process is repeated 100 times creating datasets with 90 000 samples for each combination of texture roughness and sliding direction.Data examples are shown in a matrix format in Fig. 4(b), where the piezo elements are represented by different color lines.The Niryo robot does not include force sensors on the flange and end-effector.Therefore, the VibroTact sensor was systematically placed at a fixed position of 11 mm on the z-axis to measure consistent data from all textures.The datasets created are used for training and testing methods for the recognition of texture roughness.

B. Tactile Data Processing for Roughness Recognition
In this work, machine learning (ML) methods, including KNN, SVM, ANN, and convolutional neural networks (CNN), are used and compared for texture roughness recognition with the VibroTact.
Tactile datasets, collected from all the textures, are preprocessed to extract the standard deviation feature from voltage values measured from the four piezo vibration elements in the sensor.The plots Fig. 5(a)-(e) show the bubbles that represent the standard deviation from each piezo element embedded in the VibroTact [piezo 1 (x-axis), piezo 2 (y-axis), piezo 3 (z-axis), piezo 4 (bubble size)] and each sliding direction.Each plot has a total of 800 bubbles (100 sliding repetitions×8 textures).The horizontal sliding [see Fig. 5(a)] shows a segmentation of the eight textures into two main groups, and only texture roughness T4 (10 mm bump gap) is found in both groups.The vertical sliding direction plot shows less clear boundaries between textures, and T5 and T7 (12.5 mm and 17.5 mm bump gaps) are split into different clusters.The left diagonal sliding plot shows overlap in textures T2-T5 (5 mm to 12.5 mm bump gaps) while texture T2 (5 mm bump gap) is split into two groups.Overlap in a larger number of textures is observed in the right diagonal sliding plot, and texture T4 (10 mm bump gap) is found in two clusters.In the square sliding plot, the data points are widely spread with clear overlap in textures T1, and T5-T8 (2.5 mm and 12.5-20 mm bump gaps).Data from all sliding directions and textures are shown in Fig. 5(f).
The standard deviation from voltage values from all textures are used for training and testing ML methods.KNN is the first ML method used to classify the eight artificial textures.This approach is analyzed systematically by increasing the K value and observing that K = 4 achieves the optimal performance in recognition accuracy.The highest accuracy of 100% is obtained by the horizontal sliding direction, while the lowest accuracy of 92.5% is achieved by the vertical sliding.The confusion matrices in Fig. 5(h) and (i) with high and low classification performances show that KNN misclassified mainly textures T1, T3, T6, and T8.The second ML method used for texture roughness classification is SVM.This is implemented with a Gaussian kernel achieving the highest accuracy of 100% with horizontal sliding direction and lowest accuracy of 95% for vertical sliding.Classification of individual textures by this method is presented in the confusion matrices in Fig. 5(j) and (k), with the main errors occurring in textures T1, T3, and T6.Recognition of texture roughness with the VibroTact sensor is also analyzed using ANNs.Different hyperparameter configurations were systematically analyzed to achieve the optimal recognition accuracy performance.This process found that the ANN with two hidden layers of 60 neurons each, sigmoid activation function, training with 300 epochs, and batch size of 5, achieves the best performance.The highest and the lowest recognition accuracies of 94.40% and 90.60% are obtained with horizontal and left diagonal sliding directions, respectively.Fig. 5(l) and (m) shows the resulting confusion between textures T2-T4 and T8 for the horizontal sliding and textures T1-T4, T6, and T8 for the left diagonal sliding.The fourth ML method used for texture roughness recognition is CNN.This is implemented using three convolution layers with 32, 16, and 8 kernels, respectively, and max-pooling layers for feature extraction.The output from the convolution and max-pooling processes is the input to a fully connected layer with 64 neurons, which is then connected to the output layer with eight neurons, one for each texture roughness.The CNN method achieved the highest accuracy of 99.4% with horizontal sliding procedure.Sliding in right diagonal and square directions achieved the lowest recognition accuracy of 98.10%.Fig. 5(l) and (m) shows that the main confusion occurred in texture T8 for horizontal sliding and texture T8 for vertical diagonal sliding direction.Table 1 shows the summary of the highest (in bold format) and lowest accuracy achieved by each ML method and sliding direction.In our future work, we plan to combine the tactile data from all sliding directions and used more complex texture roughness, texture material, and stimuli pattern and more ML methods for the recognition process.Overall, the results show that the VibroTact is an affordable sensor capable of measuring reliable data for recognition of a variety of texture roughness using ML methods and EPs.

IV. CONCLUSION
This letter presented the VibroTact: a soft fingertip device embedded with four flexible piezo vibration elements for recognition of texture roughness.The VibroTact device and its vibration sensing capability were tested with the recognition of eight artificial textures with different roughness properties.The sensor was mounted on the Niryo robot arm to implement five different sliding EPs for data collection.The data were processed using KNN, SVM, ANN, and CNN methods to determine the sliding direction that provide better information for texture roughness recognition.The results showed that KNN and SVM methods achieved the highest recognition accuracy of 100% when the robot implemented the sliding exploration in the horizontal direction.The lowest accuracy of 90.62% was obtained with the ANN method and sliding exploration in left diagonal direction.Overall, the VibroTact was able to recognize the eight textures using vibration with different accuracies according to the sliding exploratory direction.This affordable sensor showed to have the potential for robotic applications that involve autonomous object exploration, recognition, and manipulation.

Fig. 2 .
Fig. 2. VibroTact design.(a) Mould and Ecoflex for the soft sensor body.(b) Embedded piezo elements.(c) Sensor terminals attached to a holder.(d) and (e) Sensor dimensions and exploded illustration.

Fig. 4 .
Fig. 4. VibroTact sliding exploration and vibration data for texture roughness recognition.(a) Robot arm slides the soft fingertip sensor in five directions for data collections.(b) Vibration data examples from eight textures with different roughness (columns) for each sliding direction (rows).

TABLE 1 .
Summary of Recognition Accuracy (In %) for Each ML Method and Sliding Direction