Color Extraction and Edge Detection of Nutrient Deficiencies in Cucumber Leaves Using Artificial Neural Networks

The research aims to detect the combined deficiency of two nutrients. Those are nitrogen (N) and phosphorus (P), and phosphorus and potassium (K). Here, it is referred to as nutrient deficiencies of N and P and P and K. The researchers use the characteristics of Red, Green, Blue (RGB) color and Sobel edge detection for leaf shape detection and Artificial Neural Networks (ANN) for the identification process to make the application of nutrient differentiation identification in cucumber. The data of plant images consist of 450 training data and 150 testing data. The results of identifying nutrient deficiencies in plants using backpropagation neural networks are carried out in three tests. First, using RGB color extraction and Sobel edge detection, the researchers show 65.36% accuracy. Second, using RGB color extraction, it has 70.25% accuracy. Last, with Sobel edge detection, it has 59.52% accuracy.

Abstract-The research aims to detect the combined deficiency of two nutrients. Those are nitrogen (N) and phosphorus (P), and phosphorus and potassium (K). Here, it is referred to as nutrient deficiencies of N and P and P and K. The researchers use the characteristics of Red, Green, Blue (RGB) color and Sobel edge detection for leaf shape detection and Artificial Neural Networks (ANN) for the identification process to make the application of nutrient differentiation identification in cucumber. The data of plant images consist of 450 training data and 150 testing data. The results of identifying nutrient deficiencies in plants using backpropagation neural networks are carried out in three tests. First, using RGB color extraction and Sobel edge detection, the researchers show 65.36% accuracy. Second, using RGB color extraction, it has 70.25% accuracy. Last, with Sobel edge detection, it has 59.52% accuracy.
Index Terms-Color Extraction, Edge Detection, Nutrient Deficiencies, Artificial Neural Networks

I. INTRODUCTION
P LANTS like other living things need a combination of nutrients to grow and multiply. When plants are exposed to nutrient deficiencies, those will be unhealthy plants. The nutrient requirements of plants are divided into two categories, namely macro and micronutrients. Macronutrients are needed in large quantities. The nutrients are nitrogen, potassium, sulfur, calcium, magnesium, and phosphorus. Meanwhile, micronutrients are required in small amount, such as Received: Aug. 31, 2019; received in revised form: Mar. 16,2020; accepted: Mar. 31, 2020; available online: Apr. 22, 2020. *Corresponding Author iron, zinc, copper, and chlorine [1]. Nitrogen, phosphorus, potassium, and calcium deficiencies are determined by the plant height, the number of leaves, and the length and width of leaves. Thus, it will affect the production and quality of the fruit [2].
The research uses cucumber leaves. It has benefits as a medicinal plant for people with hypertension. Nitrogen, phosphorus, and potassium in cucumber are easily lost due to evaporation and the influence of water. Farmers usually know that cucumbers lack nutrients when fruit production has decreased.
The early detection of nutrient deficiencies is based on surface color and leaf edge shape. The characteristics of nutrient deficiency in cucumbers in the early stages can be seen from the leaf color and form on the edge of the leaf. Agricultural experts can know it. They can use their skills and knowledge to analyze plants affected by nutrient deficiencies in cucumber. However, the limited number of experts compared to the area of agriculture and plantations cannot overcome all problems related to nutrient deficiency. Conventionally, many farmers do not know the characteristic nutrient deficiency in cucumber leaves. They only realize that plants are affected by nutrient deficiencies at the severe stage resulting in losses due to crop failure.
Moreover, maintenance and identification of plants affected by nutrient deficiency are made manually. It is by observing the condition of the plant and analyzing the results. Obstacles that appear in this condition is the result of a long time manual analysis, especially for the vast plantation areas. However, the feature extraction method, namely the extraction of Red, Green, Blue Cite this article as: A. Qur'ania, P. Harsani, Triastinurmiatiningsih, L. A. Wulandhari, and A. A. S. Gunawan, "Color Extraction and Edge Detection of Nutrient Deficiencies in Cucumber Leaves Using Artificial Neural Networks", CommIT (Communication & Information Technology) Journal 14(1), 23-30, 2020. (RGB) color and Sobel edge detection for leaf shape detection and Artificial Neural Network (ANN) for the identification process, can solve the problem.
Feature extraction, which is used to differentiate deficiency, is based on color characteristics, namely red, green, and blue [3][4][5]. Meanwhile, to distinguish leaf surfaces, the feature extraction is based on leaf texture [6]. In addition to color features, object shape characteristics can be used for object identification. The shape features on leaves can be used for plant identification [7]. The method used to identify nutrient deficiencies is ANN.
The image entered through the preprocessing stage is the image that is at the cropping and resizing stage [8]. The preprocessing stage is crucial because the new image can be processed to the next stage after going through this stage. Furthermore, the image that has gone through the preprocessing stage is extracted for the color and shape features [9]. The color feature extraction used is RGB. Meanwhile, for shape extraction, it is Sobel edge detection [10]. RGB color feature extraction aims to get the RGB value of the leaves. Then, Sobel edge detection seeks to obtain the edge of the leaf so that it results in 1 and 0 value. The 1 value means the edge of the leaf, and 0 is the background of the image [9].
RGB color feature extraction is used because the color characteristics of leaves with nutritional deficiencies has color variations. Some of them are almost the same, so that the RGB color must be extracted. Thus, the values obtained from RGB can be used as a differentiator of nutrient deficiencies. In addition, Sobel edge detection is carried out because the types of leaves affected by nutrient deficiencies have different shapes. For example, the leaves that lack nitrogen will turn yellow so the photosynthetic process is not optimal.
The chlorophyll in the leaves shows the nutrient status of nitrogen in plants [11].
The color and the edge detection values are the inputs that will be transformed by the neuron layer to get the different values in the leaf image. It has a nutrient deficiency of nitrogen and phosphorus (N and P) or phosphorus and potassium (P and K) using ANN. ANN can acquire a piece of knowledge, even though there is no certainty. Generalization and extraction from certain data patterns can also be made with ANN. Therefore, ANN is suitable to be applied in the case of the identification or classification of an object.
References [12,13] have developed research using leaf imagery. All two studies result in an application that can identify early types of medicinal plants. The extraction techniques used are fractal codes and texture analysis. The resulting accuracy is 68% and 78%, respectively. Digital image processing can be used to detect disease symptoms and nutrient deficiencies that are shown through visual appearance in plant parts. The leaves are one part of the plant that is easier to analyze [6]. Based on the features obtained for plant identification, these features can also be used to detect symptoms of disease and nutrient deficiencies.
Identification of disease symptoms and nutrient deficiencies through symptoms in plant parts has been carried out by Ref. [14]. The researchers make a tool that can estimate nitrogen in rice. The results of processing through digital images are compared to results obtained through a spectroradiometer. Moreover, in Ref. [15], the detection of nitrogen and potassium is done in mustard green. Similarly, the nutrient detection system of nutrient deficiencies in wheat is made by Ref. [16].
Moreover, there is an automatic application to detect nutrient deficiencies in coffee plants [17]. Some previous research related to nutrient deficiency is limited to one nutrient deficiency. In reality, nutrient deficiencies can occur simultaneously. If there are nutrient deficiencies simultaneously, it will be difficult to distinguish because the symptoms become more complex.
Nutrient deficiencies in the two nutrients in the research is different from research that has been carried out previously. They only record plants with one nutrient deficiency. This research combines more than one nutrient to represent real events in the field that occur on cucumber leaves.
The research aims to detect the combined deficiency of two nutrients (N and P and P and K). The researchers use the characteristics of RGB color and Sobel edge detection features on the image of cucumber leaves to be an input for ANN to make the application of nutrient differentiation identification in plants.
The research uses the development of technology and computer science with the help of images that will be processed with color extraction techniques and detection of the edge of leaves to identify nutrient deficiencies in cucumber. The results can be used by users to detect nutrient deficiencies in cucumber through the image of cucumber leaves, growth, and development. Thus, it can reduce the factors of the decline in cucumber production and the possibility of crop failure. using silica sand to monitor the nutritional content of N, P, and K.
The characteristic that will be identified in the research is the combination of three characteristics of nutrient deficiencies consisting of N, P, and K. The nutrient deficiencies in plants can be recognized by the color and shape of the leaves in plants. The lack of N shows a pale green color that is usually combined with a yellowish color. Similarly, the lack of P causes the leaves to turn dark green, and some are greyish. Then, there is red pigment on the lower leaves, and the leaves will wither and die. There is also a purple color that turns to yellow on the edges of leaves, stems, and branches. Meanwhile, the lack of K causes small patches in shape. It has a pale green color causing the leaves to dry out.
The data collection is carried out by dividing the plants in 2 weeks, 3 weeks, 4 weeks, and 40 days for each cucumber that experiences normal treatment, nutrient deficiencies of N and P, and nutrient deficiencies of P and K. Data are taken in the morning and afternoon by taking pictures of the leaves (the bottom, middle, and top of the plant) with a shooting distance of 15 cm to 25 cm. It is to determine the difference in color in cucumber images. The total images are 600. Those are divided into 450 training data and 150 testing data for normal, N and P, and P and K leaf images.
Then, there are three treatments for the cucumber. First, it is the treatment for normal plants containing N, P, and K. The second treatment is for cucumbers which lack two nutrients (N and P). The third treatment is for cucumbers lacking P and K. The computer accepts the input of digital image data in the form of RGB color values, skewness, kurtosis and variance, and Sobel edge detection values [18]. Based on the input of color and edge detection values, the neural network will identify training data. It is based on calculations with training data to recognize the types of plants that have nutrient deficiencies represented by leaves through digital data.
The stages carried out by the system can be seen in Fig. 1. RGB color elements are calculated using statistical calculations, including standard deviation (Eq. (1)), skewness (Eq. (2)), and kurtosis (Eq. (3)) [19,20]: In Eqs. (1)- (3), it should be noted that M is the height of the image, N represents the width of the image, and P ij is the color value in row i and column j.

A. Image Preprocessing
The preprocessing of the leaves image is carried out so that the images can be introduced by ANN. The preprocessing process includes the stage of image cropping and selecting a significant part of the cucumber image. Then, it proceeds with the image resizing stage. In the research, the image is resized to 50 × 50 pixels so that the matrix obtained is not too many and can accelerate the process of shape extraction and training of neural networks for the extraction of shapes. The comparison of images before and after preprocessing can be seen in Fig. 2.

B. Color and Edge Detection
The RGB color extraction process in images uses Eq. (4). It shows that R is the value of the red component, G the value of the green component, and B the value of the blue component. Then, Eq. (5) is the example of formula commonly used to change to grayscale [21,22]. The equations can be seen as follows:   The results of detection in normal leaf image, nutrient deficiencies of N and P, and nutrient deficiencies of P and K deficiency are shown in Fig. 3. The next step for images that have passed the preprocessing process is the extraction of edge features using Sobel edge detection. Edge detection is to obtain the edge of an object in an image. A drastic change in the value of intensity at the boundary of two regions is utilized by edge detection.
An edge is defined as a set of connected pixels located at two boundaries [21]. Edge has very important information. Information obtained can be in the form of objects [6]. Sobel operators are more sensitive to diagonal edges compared to vertical or horizontal edges. It is different from prewitt operators. Prewitt operators are more sensitive to vertical and horizontal edges. The example of the Sobel operator in the image is shown in Fig. 4. The Sobel operator is the magnitude of the gradient calculated [5]. Moreover, the stage of edge detection using Sobel edge detection is shown in

C. Identification of Neural Networks
The identification stage is the result of identifying cucumber using the backpropagation neural network method with the number of hidden layer of 2, 4, 7, 10, 15, 20, 30, 40, and 50 and the number of epochs of 100, 500, 1000, 2000, 3000, 4000, 4000, 4000, and 5000. The identification process is done by taking the value of the color extraction and the value of the extraction form as a comparison with the training data that has previously been trained using ANN [23,24]. The identification results will appear following the training data if the value of color extraction and edge detection is close to or in accordance with the value in the training data. The color extraction results using the RGB are shown in the color histogram in Fig. 6.

III. RESULTS AND DISCUSSION
The training image is calculated using the skewness value (Eq. (1)) to find out the asymmetry. Meanwhile, kurtosis (Eq. (2)) is to find out the distribution of tapered or accumulated data. Then, variance (Eq. (3)) aims to see the distribution of values and find out the different values of the images to be an input for neural networks [20]. The examples of the calculation of skewness, kurtosis, and variance are shown in Fig. 7.
Based on the results in Fig. 7, it shows that the skewness value in the images of normal leaves and  leaves with nutrient deficiencies of P and K is clearly distinguished by the green (G) color. The skewness value of normal leaves is 1.084. Meanwhile, in leaves with nutrient deficiencies of P and K, the value is 0.658. The variance values for the three pictures show the distinguishing characteristic of green (G) with 2097 for normal images, 1935 for nutrient deficiencies of N and P, and 2532 for nutrient deficiencies of P and K. It suggests that the images with nutrient deficiencies of N and P are less green than the normal images and nutrient deficiencies of P and K images.
Moreover, based on the data on the average value of skewness, kurtosis, and variance of 450 training data with normalization values, it obtains the spread chart, as shown in Fig. 8. It shows the comparison of the values of skewness, kurtosis, and variance in normal images, nutrient deficiencies of N and P images, and nutrient deficiencies of P and K images. The normal images spread evenly with the direction of the ascending trend for the three values of skewness, kurtosis, and variance. Meanwhile, for images with Cite this article as: A. Qur'ania, P. Harsani, Triastinurmiatiningsih, L. A. Wulandhari, and A. A. S. Gunawan, "Color Extraction and Edge Detection of Nutrient Deficiencies in Cucumber Leaves Using Artificial Neural Networks", CommIT (Communication & Information Technology) Journal 14(1), 23-30, 2020.   nutrient deficiencies of N and P tend to decrease. For the value of nutrient deficiencies of P and K images, it has an increase in variance and a decrease in kurtosis compared to the skewness. This network experiment has a function to create ANN in accordance with the data to be trained. It is also to create a network with the minimum error value so that the errors during the test will be smaller, and the greater accuracy is obtained. The network test experiment is done with the number of the hidden layers of 2,4,7,10,15,20,30,40, 50 and the number of epochs of 100, 500, 1000, 2000, 3000, 4000, 4000, 4000, 5000. A test is conducted to find the smallest error values. The results can be seen in Tables I-III for 2, 4, 7, and 50 hidden layers, and 5000 epoch.
Based on Table I, it can be concluded that the determination of the backpropagation network for RGB color extraction and edge detection with the minimum error value is the third experiment. It is with the function layer of [19 7 1], which means there are 19 inputs, 7 hidden layers, and 1 output. With the learning rate of 0.00001, the performance function (Mean Squared Error (MSE)), 5000 epochs, and goal  Table II is found in the third experiment with the layer function of [7 7 1]. There are 7 inputs, 7 hidden layers, and 1 output. With the learning rate of 0.00001, the performance function (Sum Squared Error (SSE)), 5000 epochs, and goal parameter of 0.005, it has an average error value of 0.43.
The determination of the backpropagation network for Sobel edge detection with the minimum error value is the third experiment with the function layer of [12 7 1 ]. It means there are 12 inputs, 7 hidden layers, and 1 output. Using the learning rate of 0.00001, the performance function (MSE), 5000 epochs, and goal parameter of 0.005, it results in an average error value of 4.00. The test results show that the number of inputs affects the results of the mean error more than the number of hidden layers. The result can be seen in Table III.
The next stage is to obtain a backpropagation network for RGB feature extraction and Sobel edge detection form extraction. It is done by determining the target value of each type of tested leaves. The range of target values is obtained by looking at the average value of the last identification of each training data. The description of the target value can be seen in Table IV.
Next, the validation test is done by comparing the results of the training data with testing data. To calculate the accuracy of the test, the researchers use a confusion matrix. In addition, the researchers also conduct the exchange of training data with test data using k-fold cross-validation. Thus, the training data are 450 image data. Besides that, the validation test is also carried out on the data using only the extraction of color features and/or form feature extraction. For Cite this article as: A. Qur'ania, P. Harsani, Triastinurmiatiningsih, L. A. Wulandhari, and A. A. S. Gunawan, "Color Extraction and Edge Detection of Nutrient Deficiencies in Cucumber Leaves Using Artificial Neural Networks", CommIT (Communication & Information Technology) Journal 14(1), 23-30, 2020. testing data, the researchers use only color extraction or form extraction. The data exchange with five k-fold cross-validations is not carried out so that the amount of testing data used is 150 data. The results of the test are shown in Table V.
Based on the three tests that have been done, the best accuracy is obtained by using RGB color extraction with an accuracy of 70.25%. It is done with ten times testing. Meanwhile, the smallest accuracy is achieved by identification with Sobel edge detection with an accuracy of 59.52%.
The results show that differences in leaf color indicate nutrient deficiency status. It can produce the highest accuracy in accordance with previous studies seen from the visual way of leaf color [11] that the edge detection has not shown the highest level of accuracy. It is because there is no regular pattern appearance of brown spots at the edge of the leaf.

IV. CONCLUSION
The research uses the characteristics of RGB and Sobel edge detection features on the image of cucumber leaves. So, it becomes an input for ANN to identify nutrient deficiencies of N and P and P and K on cucumber leaves. Moreover, the research uses training data of normal leaves and leaves that have a combined deficiency of two nutrients (N and P and P and K). The value of skewness, kurtosis, and variance in RGB color images in normal, nutrient deficiencies of N and P, and nutrient deficiencies of P and K can be seen from the comparison of histogram values. The test results using the k-fold cross-validation with k = 5 produce accuracy values with three types of experiments. The first test using RGB color extraction and Sobel edge detection has an accuracy of 65.36%. The second test with RGB color extraction acquires an accuracy of 70.25%. Last, by using Sobel edge detection, it produces the lowest accuracy of 59.52%.
In future research, the researchers can use another feature extraction by adopting texture extraction to add features to the surface shape of cucumber leaves that have yellow and brown spots. It is recommended to investigate other features to improve accuracy by adding a ratio of leaf shape for identification.