Novel automatic scorpion-detection and -recognition system based on machine-learning techniques

All species of scorpions can inject venom, some of them even with the possibility of killing a human. Therefore, early detection and identification are essential to minimize scorpion stings. In this paper, we propose a novel automatic system for the detection and recognition of scorpions using computer vision and machine learning (ML) approaches. Two complementary image-processing techniques were used for the proposed detection method to accurately and reliably detect the presence of scorpions. The first is based on the fluorescent characteristics of scorpions when exposed to ultraviolet light, and the second on the shape features of the scorpions. Also, three models based on ML algorithms for the image recognition and classification of scorpions are compared. In particular, the three species of scorpions found in La Plata city (Argentina): Bothriurus bonariensis (of no sanitary importance), Tityus trivittatus, and Tityus confluence (both of sanitary importance) have been researched using a local binary-pattern histogram algorithm and deep neural networks with transfer learning (DNNs with TL) and data augmentation (DNNs with TL and DA) approaches. A confusion matrix and a receiver operating characteristic curve were used to evaluate the quality of these models. The results obtained show that the model of DNN with TL and DA is the most efficient at simultaneously differentiating between Tityus and Bothriurus (for health security) and between T. trivittatus and T. confluence (for biological research purposes).


Introduction
Scorpions belong to the order Scorpiones and are arthropod invertebrates that together with spiders, opiliones, pseudoscorpions, solifugae, amblypygi, mites, uropygiums, ricinulids, schizomida and palpigradi make up the group of arachnids [1]. Of all these orders, only some species of spiders and scorpions are dangerous to humans due to their venom. If we focus on scorpions, they are mainly nocturnal and usually avoid sunlight. Also, they preferably inhabit warm areas, in both wet and dry conditions, but many species inhabit temperate zones and a few live in cold zones around the world, with Antarctica being the only continent that has no registered scorpion species.
For many years, scorpions have been studied due to their high danger [2]. Given the ability of some species to inject venom with their tail stingers and their ability to kill humans and even their pets, the study of scorpions is still a topic of great interest in health care.
In Argentina, between 7000 and 8000 scorpion stings are registered per year, according to data from the Ministry of Health of this country. Also, it is estimated that 10% of cases required treatment and that the mortality is between two and eight people per year. An increase in stings usually occurs at warm times of the year, due to the increased activity of scorpions. The sectors of the population most vulnerable to the venom of a scorpion are hypertensive, cardiac or diabetic people, but also children and the elderly, even if they do not suffer from disease.
Nowadays, many scorpion populations have found new habitats provided by humans due to the modifications of the environment that did not exist a few hundred years ago, such as large cities. In particular, this can easily be observed in cities, where the good availability of food (insects, other arachnids), heat, and shelter have made cities new and appropriate sites for colonization by some species of scorpions, which are called anthropogenic.
Founded in 1882, the city of La Plata (Argentina), where this study is located, is a good example of the above. As can be seen in figure 1, two genera of scorpions are found in the La Plata city area: Bothriurus and Tityus, which are believed to have possibly arrived in the city from the middle of the twentieth century, by anthropocoria [3]. Bothriurus is a harmless genus, which is mainly found in rural and peri-urban areas (black circles), whereas Tityus is a dangerous genus that can be found in urban regions (red circles) [4].
In recent years, consultations at the CEPAVE Arachnology Laboratory (CONICET-UNLP) in La Plata city have increased, due to the increase in the appearance of scorpions in different areas of the La Plata district [5]. Specifically, there are three queried species: Bothriurus bonariensis (of no sanitary importance), and Tityus trivittatus and Tityus confluence (both of sanitary importance). These different species of scorpions can not only increase their population but can also conquer new areas.
Although Bothriurus and Tityus have many similarities, they also have some differences, especially concerning their morphology. Figure 2 shows images of the species T. trivittatus and B. bonariensis. Differences in the shapes of their tails and pedipalps can mainly be observed in this figure. It is often difficult for the human visual system to quickly recognize these different morphological features. The duality of dangerous and non-dangerous species previously mentioned generates the requirement for a quick and accurate system that can detect and identify a found specimen and assess its risk.
Computer vision is a set of computer techniques developed to interpret and process digital images, and to imitate (and even improve) the human vision system [6]. In recent years, aiming to improve their performance, machine learning (ML) techniques have begun to be used in the field of computer vision, for example, to detect, recognize, and classify objects [7][8][9][10], in face recognition [11], to analyze textures [12], in cancer detection [13], image recovery [14] and plant identification [15], among other applications [16,17]. ML is a branch of artificial intelligence whose objective is to develop techniques that allow computing units to learn [18]. These techniques are based on algorithms that convert databases into well-defined classifiers, without having to supervise the development of the latter.
In this paper, we present a novel automatic system for early detection and recognition of scorpions based on computer vision and ML techniques. The rest of this paper is organized as follows: section 2 provides a brief overview of scorpion-detection methods that have been proposed in the literature. Section 3 introduces the proposed automatic scorpion-detection system, which uses two complementary image-processing techniques: one based on the fluorescent characteristics of scorpions when exposed to ultraviolet (UV) light, and the other based on the shape features of the scorpions. Section 4 presents three ML models, which are potential candidates for use in systems recognizing and classifying scorpion images. Specifically, one model  based on the local binary-pattern histogram (LBPH) algorithm, and two models based on deep neural networks (DNNs) are described. Since DNN algorithms need a large dataset to achieve good performance, transfer learning (TL) and data augmentation (DA) approaches have been used to solve this problem. In section 5, the results from two different studies are presented and discussed. Firstly, a comparison is made of the results obtained by three ML models used to differentiate between two genera of scorpions: Bothriurus and Tityus, for sanitary purposes; and secondly, the recognition models proposed in this work are used to distinguish between two species within the genus Tityus, such as T. trivittatus and T. confluence (both of sanitary importance), for biological research and development purposes. Finally, section 6 presents the conclusions of this study. Figure 3 shows the flowchart of the proposed method in this paper, which could be implemented, as a next step, in portable devices such as smartphones and tablets. This is one of the main reasons that the system uses the TensorFlow libraries, so that it can eventually use the TensorFlow Lite tools which can be used on smartphones. Additionally, the system could be used in places without connectivity. Nowadays, the number of people with a smartphone is very high and will continue to grow over time; therefore, the proposed system has the potential to be widely distributed as a tool to improve health security.

Related work
Scorpions have been the subject of study of many researchers for a long time. Different scorpion-detection methods have been proposed in the literature. However, according to our knowledge, so far, no ML techniques have been reported in the past for identifying scorpions. The oldest methods of scorpion detection include rock rolling, burrow detection, peeling the loose bark from trees and pitfall traps, which are dangerous, time consuming and invasive [19,20]. Other methods of detection use different biological characteristics of scorpions. On the one hand, since scorpions, like other arthropods, use substrate vibration signals to recognize and locate mates and prey [21,22], an intelligent scorpion-detection system using a vibration frequency detection technique was developed in [23]. On the other hand, the cuticles of scorpions fluoresce a brilliant cyan-green (around 500 nm) under UV light [24]. This fluorescent phenomenon has long been used to detect scorpions at night, in a relatively non-invasive way, using a portable UV light [25,26]. Also, a system for automatic scorpion detection using the fluorescent characteristics under UV light was proposed in [27]; however, it should be clarified that in that work, only the green channel of the RGB image was used to perform image segmentation and therefore the idea of using UV light was abandoned for unclear reasons.
On the other hand, we previously developed an alarm system for detecting scorpions via the fluorescent principle using the H (hue) channel of the hue-saturation value (HSV) image format [28][29][30]. However, the use of UV light makes these methods efficient only at night or in the absence of daylight.
The fluorescent intensity varies widely among different species of scorpion [31]. It increases with scorpion age and with the hardness of the cuticle and it persists even after the death of the individual [32,33]. However, the role of the fluorescence has not been determined. In this regard, different theories have been proposed, e.g., that the fluorescence may help scorpions capture prey [34], attract mates, ward off predators and territorial rivals [35], identify shelter or decide when to stay in their burrows [36,37]. Also, some authors have suggested that the fluorescence may serve no behavioral purpose [32,38].

Proposed detection methodology
The automatic scorpion-detection system presented in this work is based on the implementation of a double validation process using two complementary techniques of computer-vision-based image analysis. The first technique uses a digital image-processing approach (pixel by pixel), based on the fluorescent characteristic of scorpions when exposed to UV light. A digital camera, a processing unit and a UV lamp were used for this technique. Figure 4 shows three images of the scorpion T. trivittatus under different lighting conditions and different backgrounds.
If movement is detected in part of an image, the area in question is analyzed pixel by pixel, looking for the specific fluorescent color of scorpions (a wavelength between 440 nm and 490 nm). For the color discrimination process, the HSV image format is used. If this color is detected, the presence of a scorpion can be assured. This detection system is in the process of being patented [39]. Figure 5 shows a recreated image of the scorpion T. trivittatus using the technique described.  The second technique is based on the shape features of the scorpions. In this case, a Haar cascade classifier (HCC) has been implemented in OpenCV. The HCC approach, originally used for face detection, is a fast and effective object-detection method [40]. It is an ML technique for image processing, where a lot of positive and negative images are used to train a cascade function. Then, this trained cascade function is used to detect the shape features of scorpions in other images.
A dataset of 2500 images with scorpions (positive images) and 3914 images without scorpions (negative images) has been used in this work to train the cascade function. The positive images correspond to many different genera of scorpions, in addition to Bothriurus and Tityus, whereas the negative images correspond to other type of arachnid or the absence of specimens. This training process consisted of 13 stages so as not to undergo overtraining. The system obtained was evaluated with images of scorpions, for which the correct detection was obtained in 73.3% of cases.
The images were collected from the CEPAVE Arachnology Laboratory (CONICET-UNLP), La Plata, Argentina. Additionally, images obtained from the application '¿Es araña o escorpión?' ('Is it a spider or a scorpion?') were also used [41]. Many of these positive images were taken using a live specimen, by webcam, in different situations within a controlled environment. Thus, low-quality images, without enough details to allow a clear identification or ventral (or partial) images, were considered for detection purposes but discarded for recognition purposes. Figure 6 shows a result using the HCC approach. This technique is used to complement the UV detection to avoid false detection. Moreover, although scorpions are mainly nocturnal animals, with shape-feature detection it is also possible to detect the scorpion under natural or artificial light conditions.

Recognition of scorpions using machine-learning techniques
After discarding ventral, partial and low quality images, a dataset of 132 images of two genera of scorpions (56 Bothriurus and 76 Tityus) was used to train and test three ML models in the image recognition and classification of scorpions. Specifically, 30 images of each genus were used for training (45% of the data) and the rest were used for testing purposes. All the models employed were implemented in Python. Also, an Intel Core i5 seventh-generation processor, 8 GB of RAM, and an Intel HD Graphics 620 graphic card were used. The three ML models used in this work will briefly be described below.

LBPH model
Commonly used for human face recognition, the LBPH algorithm is a simple and highly efficient texture operator, combined with histograms of oriented gradients, which greatly improves performance with some datasets [42]. To create the model to be trained, the LBPH system was implemented in OpenCV by importing the cv2 library and using the LBPHFaceRecognizer_create() function. As a first step, once the data were selected, the images were stored in an array to be used in the training phase, using the 'train' function. All the trained model data were then stored in the trainer.yml file. Before the prediction task, the images were resized to 150 × 150 pixels and transformed into grayscale. Finally, the predict(image) function returned the confidence of the corresponding prediction. The processing time for training the model was on the order of 2 min.

Model using DNN with transfer-learning
In recent years, deep-learning-based methods have been applied as novel alternatives for image recognition [43]. Deep-learning algorithms, such as DNNs or convolutional neural networks (CNNs), can automatically extract useful representative features from image data during the training process. In particular, a DNN is a feed-forward, artificial neural network, with many hidden layers between its inputs and outputs.
It is well known that deep-learning algorithms are data hungry, and produce more powerful and accurate models as more data are fed into the network. Since the number of available input data in this work (132 images of scorpions) is limited and small, the DNN training process can result in overfitting and poor performance in the test phase. A technique known as TL is usually used to overcome this limitation and to avoid training from scratch [44].
In the TL approach, the learned parameters (in particular, the weights) of networks trained well on a very large dataset are shared. The last fully-connected layers of the pre-trained model are then modified for adapting the DNN model to the scorpion classification task. Although the datasets are different, the low-level features, such as the edges, are similar. Therefore, the TL method can improve performance and reduce the training time and the processing cost, even for a small dataset. TL is used extensively in medical imaging, with great efficiency [45,46].
In this work, a sequential and dense learning model was trained using a DNN with TL. The pre-trained neural network used was VGG16 [47], which was imported from the Keras library using the TensorFlow backend. The VGG16 network is a 16-layer (convolutional and fully connected) architecture built for image recognition and classification, and based on the ImageNet dataset. In this case, resized color images were used (150 × 150 pixels for each image) to reduce the computational cost of training and evaluating the model. The network was trained for 100 epochs, with a learning rate of 0.001, using an Adam optimizer. Table 1 shows the optimized network architecture of the proposed model. The training time was less than a minute, whereas the processing time for training the model without the VGG16 pre-trained network was close to 90 min. Since the recognition problem can be considered as a dichotomous problem, the models were trained using a binary cross-entropy algorithm, which provides a Boolean response to each observation (the presence of one or another species).

Model using DNN with TL and data augmentation
Recently, in the case of small datasets, the process known as DA is another strategy that has been used to improve the training of DNNs. The purpose of this method is to significantly increase the number of images collected for the dataset, to prevent overfitting and enhance the generalization of the model [48][49][50]. Therefore, image-transformation operations, such as rotation, shearing, translation, zooming, etc., can be applied to the original dataset to produce new versions. In this work, the DA technique has been applied to the model with the VGG16 pretrained network, and implemented using the ImageDataGenerator class from Keras, with the parameters shown in table 2. Random transformation and normalization operations were done with this class during training. The number and dimensions of layers were reduced to prevent the network from getting stuck in local minima. Table 3 shows the optimized network architecture of the proposed model. In this case, the network was trained for 20 epochs, with a learning rate of 0.001, using an Adam optimizer. Also, the batch size and the steps per epoch were set to 30 and 100, respectively.

Metrics for model evaluation
To evaluate the performance of the classification models developed, a confusion matrix is used [51]. In this work, a true positive (TP) occurs when the genus Tityus is expected and it is correctly classified by the decision system. A false positive (FP) occurs when the system wrongly identifies the genus Bothriurus as the genus Tityus, whereas a false negative (FN) occurs when the system fails to recognize the genus Tityus. Figure 7 summarizes the confusion matrix used in this paper. Based on the results of the confusion matrix, four metrics are used in this study: accuracy (A), precision (P), recall (R) and F1 measure . The following equations are used to calculate these metrics [52]: Due to the danger caused by the genus Tityus, the values of FN correspond to the case with the greatest sanitary importance. Since the value of R decreases as the number of FN increases, R is the metric of main interest in this work.
Additionally, a receiver operating characteristic (ROC) curve is used to examine the behavior of the binary models for different values of the classification threshold. This curve indicates how well the classifier system can be specific and sensitive simultaneously over a range of measurements [53]. In the ROC curve, the TP rate (sensitivity) is plotted against the FP rate (1-specificity). Every point corresponds to a given instance of the confusion matrix and represents a relative trade-off between TP and FP.

Recognition and classification of two genera of scorpions: Bothriurus and Tityus
In figure 8, an example of the detection and classification of two genera of scorpions (Bothriurus and Tityus) can be observed. In this figure, the detection and classification were achieved using the HCC and the LBPH systems, respectively.
In this section, we present comparisons of the results of the classification models developed in this work for differentiating between two genera of scorpions (Bothriurus and Tityus). The confusion matrices obtained during the testing of all three models are shown in figures 9-11, where the vertical axes correspond to the true data and the horizontal axes correspond to the predictions of the models. Table 4 shows the values of accuracy, precision and recall calculated using equations (1)-(3), respectively. As can be seen, the performance of the LBPH model is lower than those achieved using the DNN models. Although the LBPH model has good precision (comparable to the two best models presented here), the value of R associated with the number of FNs (Tityus identified as Bothriurus), is compromised.
A substantial improvement is achieved with both DNN models. In particular, the DNN model without DA achieves the highest recall (0.93); however, a larger number of epochs were required to achieve this result. Values of R close to 0.9 or even higher (due to lower probabilities of FN) provide efficient recognition systems for health security, due to their great ability to identify the genus Tityus, of sanitary importance. Although these models have limitations for the identification of the genus Bothriurus, as previously mentioned, this is a harmless genus (of no sanitary importance). This limitation is related to the fact that the  number of FPs is comparable to the number of true negatives. This result has an impact on the values of A and P, which are close to 0.8.
A more detailed analysis of the three models has been made based on ROC curves, as is shown in figure 12. It can be seen in this figure that the areas under the ROC curve for the DNN models are very similar and much higher than that of the LBPH model. On the other hand, for values of FP rates lower than 0.3 and 0.4 (specificities greater than 0.7 and 0.6, respectively), the LBPH model and the 'DNN with TL and DA' model have higher sensitivity (TP rate) than the 'DNN with TL' model. Conversely, the 'DNN with TL' model has the best sensitivity when the values of the FP rate are higher.
From these results, it is possible to establish that both DNN models can be used quickly and accurately for the recognition and classification of dangerous (Tityus) and non-dangerous (Bothriurus) genera of   scorpions. However, the 'DNN with TL and DA' model is considered the best model, since the image-augmentation technique ensures a robust behavior for a significant number of training images. In the specific case of this model, for each training phase, the images generated with the DA technique have random characteristics, which cause variations in the calculated metrics. Therefore, 30 training runs were performed to study the efficiency of the proposed method. Figure 13 shows the distribution of the results obtained for

Recognition and classification of two species of Tityus scorpion
As previously mentioned, within the genus Tityus, two species can be found in La Plata city: T. trivittatus and T. confluence, both of sanitary importance. These species are relatively similar to each other, as can be seen in figure 14. The most significant difference between the two species is the three stripes that can be seen on the back of the T. trivittatus. Other minor differences can be practically imperceptible to the hion system. For research purposes, the three ML models used in this work were trained for specific identification of both species. In this case, a dataset of 76 images of the genus Tityus (31 trivittatus and 45 confluence) was used to train and test the developed models.
The LBPH model and the DNN model with TL but without DA were not able to differentiate T. trivittatus from T. confluence. However, the model with the DA approach was able to adequately identify both species, showing the utility of this technique. The optimized network architecture is the same as that shown in table 3. The network was trained for 50 epochs. The batch size was set to 30 and the steps per epoch were set to 10, therefore 300 images were used to train the model per epoch.  The confusion matrix and the ROC curve obtained during testing are shown in figures 15 and 16, respectively. In this case, a TP occurs when a T. confluence scorpion is expected and it is correctly recognized by the classifier; an FP occurs when a T. trivittatus scorpion is expected, but it is wrongly identified as T. confluence; whereas an FN occurs when the system fails to recognize a T. confluence scorpion. Values of A = 0.72, P = 0.83 and R = 0.74 were obtained for this classifier, for the accuracy, precision, and recall, respectively. Although both species are dangerous to human health, the results presented in this section are particularly useful for biological research and development purposes because they can be used to distinguish between two very similar species. Therefore, accuracy is the most relevant metric to be considered in this case. Additionally, to study the efficiency of the proposed method, a procedure similar to that previously described in section 5.1 was used. The results, in terms of a box plot, are shown in figure 17.
The accuracy results obtained in this paper agree very well with those reported in [54,55]. In those works, maximum values of accuracy close to 72% and 78% were obtained using CNN architectures to identify dog breeds [54] and dogs and cats [55].

Conclusions
In this work, a novel automatic system capable of detecting and identifying scorpions found in La Plata city (Argentina) was proposed, using computer vision and ML methods. The fluorescent property of scorpions under UV light and their shape features were used for the detection system of these arachnids through image processing techniques. On the other hand, a comparison of three different ML approaches for the image recognition and classification of scorpions was discussed in this paper. Despite having a small data set, the two models based on 'DNN with TL' and 'DNN with TL and DA' techniques, for differentiating between dangerous (Tityus) and harmless (Bothriurus) genera of scorpions yielded recall values of 93% and 89%, respectively. The performance of the two DNN models was better than that of the model commonly used for human face recognition, LBPH. As an additional result, the 'DNN with TL and DA' model was the only one capable of distinguishing between two species within the genus Tityus, such as T. trivittatus and T. confluence (both of sanitary importance), with an average precision of 81% and an average accuracy of 76%. These good predictive results are very helpful in contributing to the detection and identification of scorpions, both for health security (the distinction between Tityus and Bothriurus) and for biological research purposes (the distinction between T. trivittatus and T. confluence).

Data availability statement
The data that support the findings of this study are available upon reasonable request from the authors.