Tomato Pests and Diseases Classification Model Based on Optimized Convolutional Neural Network

In the field of agricultural information processing, automatic identification and diagnosis of common diseases of tomatoes play an important role. The emergence of deep learning can help people simplify the process of image feature extraction, reduce network complexity and improve recognition accuracy. This paper built the Inception-v3 model to identify and classify five classes of tomato pests and diseases. We collected some images of tomato diseases uploaded by farmers from the online diagnosis platform for crop pests and diseases. In the experiment, the combination of the moving average model and the regularization was used to adjust the parameters in the model and enhance the robustness of the model on the test data. Experiments showed that the Inception-v3 model based on regular expression and moving average optimization can effectively avoid regularization and has higher accuracy, which can reach 86.9%.


Introduction
The continuous development of the economy and society has brought about global climate and environmental problems. The occurrence of diseases and the variation of fungal bacteria affect people's lives. The incidence of crop diseases and pests is getting higher and higher, and the diseases are becoming more and more complicated. Therefore, it is particularly important to study the prevention and diagnosis of crop diseases. The traditional methods of artificially detecting pests and diseases depend entirely on the observation experience of the farmers, or ask experts to come to the door. Such methods are slow, inefficient, expensive, subjective, low-accuracy, and time-insensitive. With the continuous development of the Internet, the applications of information technology have provided new methods and ideas for crop pest and disease identification. A correct diagnosis is essential in order to define strategies of management and control, and consequently for the rational use of fertilizers and pesticides. One main obstacle towards a quick and accurate diagnosis system is the need for experts, making it costly to cover large domains and give a feasible solution in time. Moreover, experts often specialize in specific issues, increasing the rate of misdiagnosis. Some attempts have been tried to reduce the dependency on experts. Expert Systems, often built on top of Case-Based Reasoning algorithms, are applied to some culture. These systems still require considerable amounts of training and are often not accurate, mainly due to the typically very large number of questions required to be answered by experts and the sensitivity to the wrong answers. Using efficient image recognition technology can improve image recognition efficiency, reduce costs, and improve recognition accuracy. For this purpose, a large number of researches have been conducted by experts and scholars at home and abroad Deep learning has become the focus of future research.

Related works
Many technologies have been applied to the identification and detection of crop diseases and insect pests, especially image recognition technology, and great progress has been made in the field of pest and disease [1][2]. Image recognition is a process of converting an image signal into a corresponding digital signal and processing it by a computer, which is also called image feature extraction. Many research scholars use the automatic recognition technology based on feature extraction to identify crop pests and diseases, and the effect is remarkable [3][4][5]. In 2007, Sammany M et al. combined the two methods of support vector machine and neural network to identify plant diseases, and optimized the feature parameters with rough sets to improve the classification accuracy [6]. In 2006, Hinton et al. proposed deep learning in Science [7]. Its emergence has provided a new way to solve image recognition technology. Many researchers at home and abroad have applied deep learning methods to the detection of diseases and pests and achieved good results, paving the way for further research on the automatic identification of pests and diseases [8]. In 2012, Krizhevsky et al. used CNN called AlexNet to achieve the best results in the ImageNet competition image classification task, which is the great success of CNN in large-scale image classification [9]. In 2019, ZHANG et al. proposed a multi-featured weighted DenseNet model (MFR-DenseNet) for image classification [10]. This model improves the expressive power compared to the traditional DenseNet by automatically adjusting the channel characteristic response and the dependence between different convolutional layer features. In order to achieve the goal of calibrating the dynamic channel characteristics, the researchers introduced the Extrusion Excitation Module (SEM) in DenseNet, and then proposed the use of the Double Extrusion Excitation Module (DSEM) to simulate the dependence between different convolutional layer features. The MFR-DenseNet model was designed by combining this method with the integrated learning method. The experimental results showed that the classification effect of this model is obvious. In 2017, RAMCHARAN et al. Applied deep convolutional neural networks in cassava pest identification [11]. The experimental results showed that the recognition accuracy of cassava brown spot is 98%, the accuracy of identifying red spider is 96%, the accuracy of identifying green carp is 95%, the accuracy of identifying cassava brown streak is 98%, and the accuracy of identifying cassava mosaic disease At 96%, the experimental results showed that the depth learning model has high recognition accuracy. In 2018, Zhang et al. designed an 11-layer LeNet convolutional neural network to identify cucumber diseases [12]. The researchers collected 1200 color images of cucumber disease for cropping and normalization preprocessing, then trained the preprocessed color image and adjusted it using the RGB color channel. Experiments showed that the accuracy of the method is over 90%, which is higher than the traditional method.

Data set
The Agricultural Doctor Online Diagnostic System is an open agricultural information service platform. Crop growers can interact with relevant experts and scholars to upload problems in the process of planting crops to the platform in the form of pictures and texts. Crop pests and diseases experts will diagnose diseases based on uploaded pictures or texts. The image data set used in this article was collected from the website of Agricultural Doctor Online Diagnostic System. Due to the limited image data collected, the amount of training data required for deep learning cannot be satisfied, which can easily lead to over-fitting. Therefore, it is necessary to expand the image before the image is uniformly processed. In this paper, the training data is expanded by a combination of rotation, distortion, and rotational distortion, and the rotated image is rotated at different angles or to different degrees as a new picture. The expanded data set has 20973 images, including 5904 pictures of whitefly, 5889 pictures of gray mold, 4932 pictures of cotton bollworms, 2260 pictures of viral diseases, and 1988 pictures of late blight. We Randomly selected a picture and selected two angles for rotation, distortion. The original picture and the transformed sample picture are shown in Figure 1 It reduces the network parameters while increasing the depth of the network, and reduces the calculation while increasing the feature expression ability.

CNN model based on regularization and moving average model (1) Regularization
In the process of model training, there is often a phenomenon of over-fitting. Overfitting refers to the phenomenon that the model predicts good results for known data, but predicts poor results for unknown data. Regularization is the addition of a priori condition to the model parameters, which reduces the complexity of the model and reduces the input disturbance to noise and anomalies. Therefore, the L2 regular expression was used in this paper to effectively avoid the occurrence of overfitting. L2 regularization adds a regularization term to the loss function. The expression is: (2) Moving average In the sliding average model, the variable decay rate is set to control the speed of the model update. The num_updates parameter in the moving average model can be used to dynamically set the size of the reduce. The decay rate expression for each use is: According to experience, decay is usually set to a number very close to 1, such as 0.999 or 0.9999. In this experiment, set the value of decay to 0.999. When training the neural network model, it is necessary to update the parameters in the neural network. Using a moving average can increase the robustness of the model to test data. Robustness is the resistance to mutations. The better the robustness, the stronger the resistance of this model to malignant parameters. The moving average model can effectively reduce the impact of noise in the training data on the model.

Results
(1) Inception-v3 model: step=4000 The number of iteration step was set to 4000. Then experiment was completed under tensorflow platform, and accuracy of validation data and test data with different iterations was illustrated in Figure 5. The final accuracy of the model constructed in this experiment reached 92.1% on the test data set, but it can be seen from the line graph that the verification data set fluctuates with the increase of the number of iterations in the later stage, and the amplitude is large. This means that the number of iterations is not enough.
(2) Inception-v3 model: step=8000 Based on the results of the previous experiment, we modified the number of iterations to 8000 and again recorded the accuracy of the validation data set and test data set after each iteration. The accuracy of verifying the data set and test data set with the number of iterations was shown in Figure  6. The final accuracy of the model constructed in this experiment reached 96.2% on the test data set, and the accuracy of the verification data set will fluctuate with the number of iterations in the later stage, but the fluctuation range was small, which indicates that it is more appropriate to set the step to 8000 in the experiment. However, the accuracy of the verification data set was stable and slightly declined in the later iteration process, and the accuracy of the test data set rose steadily in the case of verifying. Therefore, Overfitting occurred during the training of the model.
(3) Inception-v3 model: adding regularization Although the second experiment was more reasonable in the setting of the number of iterations, there was a phenomenon of overfitting. Taking the lessons of the second experiment, this experiment set the step to 8000 and added a regular expression to effectively avoid over-fitting. The accuracy of the validation data set and test data set after each iteration was recorded. The accuracy of the validation data set and test data set after adding a regular expression varies with the number of iterations as shown in Figure 7. can only reach 83% on the test data set, and the accuracy is not high. Therefore, the optimization algorithm is needed to enhance the robustness of the model.
(4) Inception-v3 model: adding regularization and moving average model It can be observed that regularization can efficiently avoid occurrence of overfitting in the third experiment. But it is poor for performance of Inception-v3 model on train set which were researched in this paper. On basis of failure of the third experiment, regularization and moving average model were adopted to optimize Inception-v3 model, and validation accuracy and test accuracy after each iterating were recorded. Figure 8 is plotted to describe the trend of validation accuracy and test accuracy with different iterations.

Figure 8. validation accuracy and test accuracy
The final test accuracy is 86.9%, which is about 3.9% higher than the third experiment. Then test accuracy of the third experiment is compared with the fourth experiment. It can be seen from Figure 9, the test accuracy of model optimized by regularization and moving average was stable, and test accuracy of model optimized by regularization fluctuated greatly. The figure showed that regularization and moving average can improve accuracy and enhance robustness within certain range.

Conclusion
In this paper, four experiments were carried out. Comparing the results of the four experiments, it can be found that in the process of constructing the CNN model of image classification, not only the network structure is the key, but also the variables such as the number of iterations need to be dynamically adjusted according to the experimental results. In this paper, Inception-v3 convolutional neural network was used for image recognition. The regularization and sliding average were used to optimize the model. The accuracy of the model in the test data set reached 86.9%. Experiments showed that the image recognition accuracy of this model is higher than the traditional model. Therefore, regularization and sliding averaging can be used to optimize the CNN model, effectively improve the recognition rate of the model on the test set, and enhance the robustness of the model. This study provided ideas and directions for the research of intelligent identification and diagnosis of crop diseases and pests. The research on efficient pest and disease identification algorithm can improve the recognition efficiency of tomato pests and diseases.