1 Introduction

A growing body of studies recognizes the importance of the honeybee economy and production [1]. The statistical studies showed that the worldwide market of honeybees was valued at about eight billion U.S. dollars in 2020. In addition, the total number of global beehives reached about 94 million beehives worldwide from 2010 to 2020, increasing from about 80 million beehives in 2010 [2]. Many countries of the world are interested in the bee honey industry as an important source of national income for these countries. India is counted as one of the top leading countries in honeybees economy where it has the greatest number of beehives, totaling about 12.2 million, while China is ranked second with nine million beehives. However, China was ranked the top honey-producing country in the world. Its production volume amounted to around 458 thousand metric tons of honey in 2020. The global prediction of the honeybee economy and production is forecasted to reach about 12 billion U.S. dollars by 2028 [2].

Beehives' health is a major condition, which has a considerable impact on honeybees' production and economy. The beehive needs ideal conditions inside/outside the hive to produce new bees and keep the queen healthy enough to survive and keeps the honey production process going efficiently in that hive. Many variables can affect beehives' health such as the temperature inside and outside the hive, bees-flight activities in and around the hive, humidity inside the hive, acoustics or “buzzing” inside the hive, etc.

A sensor system that detects fluctuations in these variables can give beekeepers early alarms of problems and actions within the hive [3]. Therefore, Artificial Intelligence (AI), computer vision, and the Internet of Things (IoT) can help in developing warning and prediction systems for analyzing beehive health and monitoring bee behaviors [4]. For instance, Inside the hive, the Microphone can pick up all types of nearby sounds to the hive, then machine learning models can be used to distinguish and separate these irrelevant noises from the bees-buzzing of the hive. This methodology can assist beekeepers in analyzing bee modes and the health status of the bee colony. Monitoring honeybee flight activities is another important issue in which computer vision techniques can provide untraditional solutions to evaluate the health and strength of beehives [5]. Robust Principal Component Analysis (RPCA) [6] can be utilized as a computer vision technique to estimate an accurate count of bees and their flight activities after removing irrelevant background images from the input bee images. The estimated amount of flight activities from the hive can tell beekeepers about foraging activity since bees that exit and return to the hive are typically foraging for pollen, nectar, resin, and water. These flight activities give indicators of hives' health. Monitoring bees' flight activities can also help in tracking bee swarm formation, which is an important bee health indicator, because the sick bees cause the rest bees in the swarm to escape.

Analyzing beehives' health can be also estimated based on recognizing three problems inside beehives, Varroa destructor and hive beetles, ant problems, and missing queen. Varroa destructor [7] is a parasitic mite that attacks and feeds and reproduced on honeybees. It attaches to the body of the bee and weakens the bee by sucking fat bodies and killing it, this normally impacts the beehive's health. Attacking beehives with some species of ants can cause further problems for beekeepers. A healthy hive can easily defend itself against a few ants, but when there are swarms of ants in a beehive, the bees may escape by filling up with honey and leaving the hive. For example, Special species of dangerous ants such as Argentine ants can behave as a carrier of an insect virus known as Deformed Wing Virus (DWV), which is the cause of honeybee hives' disorder and collapsing [8]. Queen Loos from the beehives is another problem that may affect to beehives' health. Queenless hives minimize the average bee's population size, the average brood quality, and the average number of honey frames. These problems are enough to make those hives unhealthy [9].

In response to those problems of beehive health, this paper introduces a proposed deep learning methodology for analyzing beehive health. A proposed update of the Mobile Net deep learning model is implemented and validated on a benchmark dataset for classifying beehives according to three hive health impacts, Varroa destructor, ant problems, and losing the queen. Moreover, the frontend layers of the mobile net model are updated with three optimization algorithms, Adam optimizer [10], Nesterov-accelerated Adam (Nadam) Optimizer [11], and Stochastic gradient descent (SGD) [12] for performing the feature selection phase on the used dataset. The experimental results proved the efficiency of the mobile network with Adam optimizer in classifying beehives according to the three beehive health impacts where the model achieved testing accuracy of 95% and testing loss of 35%. The comparison results confirmed the superiority of Mobile Net using ADAM optimizer in recognizing beehive health abnormalities compared to four deep learning models, Shuffle Net, Resent 50, VGG-19, and Google Net.

The remaining sections of this paper are organized as follows: Sect. 2 discusses the related work, Sect. 3 presents the proposed deep learning methodology, Sect. 4 presents the experimental results, Sect. 5 discusses the obtained results, and finally Sect. 6 concludes this work.

2 Literature Review

For many years, artificial intelligence researchers surprisingly neglected the phenomenon of analyzing beehive health as a research problem. However, the literature has highlighted some works and contributions in this area.

Edwards-Murphy et al. [13] introduced an IoT monitoring system based on wireless sensor networks for collecting various beehive data (i.e. weather data, O2, CO2, temperature, humidity, etc.) to conduct a biological analysis used to classify ten important beehive states. The authors applied and validated the decision tree algorithm for describing the beehive state using sensor network data with 95.38% accuracy. Moreover, the authors studied the correlation between beehive data and meteorological conditions using a prediction algorithm to forecast short-term rainfall based on the beehive's parameters. The validation results showed high prediction accuracy (95.4%), low power, and computational overhead (5.35% energy consumption).

Cecchi et al. [14] developed a smart heterogonous sensor network system for long-term measurements and real-time monitoring of beehives. The study was aiming to collect some beehive parameters related to beehive features, such as bees-sounds, the hive weight, humidity, temperature, and CO2 inside the beehive, as well as weather conditions outside the beehive. The analysis of the collected data about those parameters through the developed sensor network clarified the efficiency of the proposed system as a monitoring system for monitoring beehive health.

Braga et al. [15] developed a machine learning-based method to predict beehive health based on training and testing three well-known classification algorithms (k-Nearest Neighbors, Random Forest, and Neural Networks). These algorithms have been validated using real datasets from 6 apiaries, and 27 Western honeybees (Apis mellifera) beehives monitored over 3 years (2016, 2017, and 2018). Training and testing data have been built on a set of beehive features [e.g. beehive weight, temperature as well as weather data (dew point, temperature, wind speed, and direction, rainfall, and daylight)] to predict the health status of honey beehives. The experimental results clarified that beehive health prediction based on the Random Forest algorithm achieved a prediction accuracy of over 90%.

Andrijević et al. [16] developed additional IoT and AI-based monitoring and prediction system for beehives. The authors developed a sensor network to collect a wide range of parameters inside the beehive and a bee counter at the hive entrance, then three time-series forecasting models (Autoregressive Integrated Moving Average (ARIMA), Facebook Prophet, and Long short-term memory (LSTM) networks) have been validated to monitor beehives health based on estimating the volume of bee exits and entrances. The comparison results showed that the LSTM model achieved the best forecasting results with the lowest root mean square error (RMSE) (426.49 for the bee-out time series, and 378.46 for the bee-in time series) compared to ARIMA and Facebook Prophet models.

Berkaya et al. [17] introduced a deep learning model to monitor beehive health by recognizing various conditions as well as abnormalities from the bee images dataset. The authors applied a pre-trained transfer deep learning neural network (DNN) model for recognizing abnormal conditions such as ant problems, Varroa parasites, hive robberies, and small hive beetles, which make beehives unhealthy colonies. Moreover, the support vector machine model with shallow features, deep features, and both shallow and deep features extracted from the DNN model has been implemented and validated. The proposed models have been trained and tested on three real datasets consisting of 19,393 honeybee images. The validation results clarified that the proposed model could recognize beehive health abnormalities with an accuracy of up to 99, 07%.

Other studies analyzed beehive health based on bees' sound classification. Bee sounds enable bee swarms to communicate within the hive, and their sound analysis can reveal important information to interpret the colony's health status and detect sudden abnormalities using an acoustic system [18].

Zgank [19] proposed IoT-based bee activities sound classification system to detect swarming conditions, which is a great concern to beekeepers. Swarming means the hive has become overcrowded and bees are reproducing. Therefore, this is an important indicator to beekeepers about the honeybee production process and beehive health. Without swarming, this means that there is a lack of resources, disease, or some kind of disturbance in the hive. The validation results clarified that a good sound classification performance can be achieved with the proposed IoT system based on Mel-Frequency Cepstral Coefficients and Linear Predictive Coding. Moreover, Different Hidden Markov Models and Gaussian Mixture Models' topologies proved their efficiencies in determining the most suitable acoustic model for the proposed IoT system.

Nolasco et al. [20] investigated the efficiency of both convolutional neural networks and support vector machines for recognizing beehive states based on queen loos. The two models have been validated and tested using an acoustic dataset of beehives gathered from the NU-Hive project [21]. The validation results clarified the efficiency of both two models in recognizing queen absence based on bee sound classification.

A similar study conducted by Guerrero et al. [22] studied the problem of a queenless beehive and its impact on bee hives' health based on bee sound classification. The authors investigated the efficiency of Lasso Logistic Regression and Singular Value Decomposition to recognize bee sound patterns in queenless beehives. The validation results proved that healthy and queenless hives can generate slightly different sound patterns based on monitoring five hives of the Carniola honey bee.

In all the studies reviewed here, beehives' health analysis and investigation are recognized as a hot topic in bee honey production. However, a few studies discussed the feasibility of developing efficient deep learning models for solving this problem based on the following features, missing queen, Varroa destructor, hive beetles, ant problems, and hives robbing. Therefore, this study may fill this gap of knowledge in this research area.

3 Analyzing Bees Hives’ Health: Deep Learning Methodology

Some deep learning methods have been proposed to solve many beehives' health classification problems based on various features, which affect beehives' health state. A proposed deep learning model is proposed to recognize a set of beehives' health abnormalities based on the following features, missing queen, Varroa destructor, hive beetles, ant problems, and hives robbing. The proposed deep learning approach is designed based on two major phases: the first phase performs the feature selection using three optimizers to get the best features that influence the classification accuracy. The adopted optimizers are the Adam optimizer, Nesterov-accelerated Adam (Nadam) optimizer, and stochastic gradient descent (SGD). The optimization methodologies of the three optimizers are specified in Algorithms 1, 2, and 3, respectively. Mathematically, the optimization methodology of feature selection can be specified by calculating the objective function, f(θ), which is used to evaluate the results of the parameter, \(\hat{\theta }\). The main objective of f(θ) is minimizing the error. With computing the gradient of unknown parameter vector \(\hat{\theta }\) as in Eq. (1), we can compute the error with new values of \(\hat{\theta }\) according to the methodology of each optimizer (i.e. SGD, ADAM, or NADAM,).

$$g_{t} = \nabla_{\theta } f_{t } (\theta ).$$
(1)

According to the steps of the SGD algorithm, we can compute \(\hat{\theta }\) by using Eq. (2):

$$\theta_{t} = \theta_{t - 1} - \gamma \times \nabla_{\theta } f_{t} (\theta ;\;x^{i} ;\;y^{i} ),$$
(2)

where θ is a vector of parameters, it is dependent on the learning rate only. While ADAM and NADAM use other parameters for updating θ. X is the vector of inputs and Y is the vector of outputs.

ADAM optimizer uses Eq. (3) to get θ:

$$\theta_{t} = \theta_{t - 1} - \gamma \times \widehat{{m_{t} }}/\left( {\sqrt {\widehat{{v_{t} }}} + \epsilon } \right),$$
(3)

where \(\epsilon\) is the learning rate, \(\gamma\) is a Step-size, \(\widehat{{m_{t} }}, \;{\text{and}}\; \widehat{{v_{t} }}\) are initialization vectors. The parameter vector θ depends on two updated values bias-corrected first-moment estimate \(\widehat{{m_{t} }}\) and bias-corrected second raw moment \(\widehat{{v_{t} }}\). They are updated by using Eqs. (4), (5), respectively.

$$\widehat{{m_{t} }} = m_{t} /(1 - \beta_{1}^{t} )$$
(4)
$$\widehat{{v_{t} }} = v_{t} /(1 - \beta_{2}^{t} ),$$
(5)

where β1 and β2 ∈ [0, 1] rates of exponential decay.

In the NADAM algorithm, we compute the gradient at each batch and update vector θ as written in Eq. (6):

$$\theta_{t} = \theta_{t - 1} - \gamma \times g_{t} .$$
(6)

NADAM takes more time than SGD and ADAM because it has many computations and update parameters at each batch.

In the second phase, the Mobile Net model [23] is utilized to perform the classification task on the obtained optimized features. Figure 1 explains the general architecture of the proposed approach. In addition, Fig. 2 explains the methodology of the Mobile Net model. It consists of 28 layers with total parameters: of 3,235,014 parameters, trainable parameters: of 3,213,126, and non-trainable parameters: of 21,888. MobileNet methodology depends on two computation layers, a depthwise separable convolution layer with filter size 3 × 3 and a pairwise convolution layer with filter size 1 × 1, this architecture speed-up response time of model learning. The call of those two layers is done after performing the feature selection phase using one of the selected optimizers, ADAM, NADAM, and SGD. Finally, the CNN architecture consists of convolutional layers with 32 filters, pooling layers with 64, fully connected layers with 128 filters, and 1 dense layer with six neurons for classification results.

Fig. 1
figure 1

The general architecture of the proposed approach to classify beehive health abnormalities

Full size image
Fig. 2
figure 2

The deep learning methodology of the Mobile Net model to classify beehives health abnormalities

Full size image

To test the efficiency of the proposed deep learning approach, a benchmark dataset [24] is used to perform the two phases, feature selection optimization, and beehives classification.

figure a
figure b
figure c

4 Experimental Results

To evaluate the proposed deep learning methodology in classifying beehive health abnormalities, a benchmark dataset consisting of 5172 bee images [24] has been used to validate the performance of the proposed deep learning methodology of the Mobile Net model. This dataset has six classes of abnormalities that negatively impact the beehives' health, including ant problems, few Varroa-hive beetles, healthy, hive being robbed, missing queen, and Varroa-small hive beetles. The proposed methodology has been developed according to the implementation attributes in Table 1.

Table 1 Implementation environment
Full size table

The confusion matrices results of calculating True positives (TP), True negatives (TN), False Positives (FP), and False negatives (FN) of the six abnormalities classes in the used dataset are depicted in Fig. 3. Evaluating the optimization performance of Mobile Net methodology using ADAM, NADAM, and SGD algorithms to get the best features used to classify honeybees abnormalities are depicted in Figs. 4, 5, and 6, respectively. In addition, the validation loss and validation accuracy results of the Mobile Net methodology using ADAM, NADAM, and SGD algorithms in classifying honeybee's abnormalities are depicted in Figs. 7, 8, and 9, respectively.

Fig. 3
figure 3

Confusion matrices results of the Mobile Net using three optimizers: a ADAM, b NADAM, and c SGD

Full size image
Fig. 4
figure 4

Classification results of the Mobile Net methodology using ADAM optimizer

Full size image
Fig. 5
figure 5

Classification results of the Mobile Net methodology using NADAM optimizer

Full size image
Fig. 6
figure 6

Classification results of the Mobile Net methodology using SGD optimizer

Full size image
Fig. 7
figure 7

Training and validation loss and accuracy of the Mobile Net methodology using ADAM optimizer

Full size image
Fig. 8
figure 8

Training and validation loss and accuracy of the Mobile Net methodology using NADAM optimizer

Full size image
Fig. 9
figure 9

Training and validation loss and accuracy of the Mobile Net methodology using SGD optimizer

Full size image

5 Discussion

Very little was found in the literature on the challenge of developing efficient deep learning models for recognizing beehive health abnormalities problems based on the following features, missing queen, Varroa destructor, hive beetles, ant problems, and hives robbing. The present study was designed to determine the efficiency of using the Mobile Net as a deep learning methodology for recognizing beehive health abnormalities based on utilizing three optimizers, ADAM, NADAM, and SGD to select the best features used to train and test the Mobile Net methodology. It is interesting to note that in the three cases of applying Mobile Net methodology based on three optimizer techniques, ADAM, NADAM, and SGD, the Mobile Net using ADAM optimizer achieved the best validation accuracy (95%) and the lowest validation loss (35%) in recognizing beehives health abnormalities as depicted in Fig. 7. Moreover, Mobile Net using ADAM optimizer achieved the best precision and recall results while recognizing the four types of beehive health abnormalities as depicted in Figs. 4, 5, 6.

On the other hand, the optimization performance of Mobile Net using NADAM, and SGD achieved validation accuracy, of 89%, and 76% as depicted in Figs. 8, and 9 respectively. Figure 10 compares the validation accuracy and validation loss of Mobile Net using the three optimizers, ADAM, NADAM, and SGD which confirm the superiority of the ADAM optimizer in performing the feature section phase of the proposed Mobile Net model.

Fig. 10
figure 10

Validation accuracy and validation loss of Mobile Net using ADAM, NADAM, and SGD optimizers

Full size image

To validate the obtained results of this study, four experiments have been conducted on the used dataset. The three optimizers, ADAM, NADAM, and SGD have been practiced with four deep learning methodologies, Shuffle net [25], Resnet 50 [26], VGG-19 [27], and Google Net [28]. The comparison results confirmed also the superiority of Mobile Net using ADAM optimizer in recognizing beehive health abnormalities compared to the selected four deep learning methods, Shuffle Net, Resent 50, VGG-19, and Google Net. Figure 11 compares the validation accuracy, and Fig. 12 compares the validation loss between Mobile Net and the four deep learning methods, Shuffle Net, Resnet 50, VGG-19, and Google Net using the three optimizers, ADAM, NADAM, and SGD.

Fig. 11
figure 11

Comparing validation accuracy of Mobile Net with Shuffle Net, Resnet 50, VGG-19, and Google Net using the three optimizers, ADAM, NADAM, and SGD

Full size image
Fig. 12
figure 12

Comparing validation loss of Mobile Net with Shuffle Net, Resnet 50, VGG-19, and Google Net using the three optimizers, ADAM, NADAM, and SGD

Full size image

The possible explanation for these results confirms that ADAM optimizer tends to converge faster than NADAM and SDG while finding the optimal features used to recognize the four types of beehives health abnormalities (i.e. Varroa destructor and hive beetles, ant problems, and missing queen), hence, the classification performance of the MobileNet using ADAM optimizer achieved the best accuracy, precision, recall, and lowest validation loss while recognizing the four types of beehives health abnormalities.

6 Conclusion

The present study was designed to determine the efficiency of using the Mobile Net as a deep learning methodology to recognize beehive health abnormalities. The performance of Mobile Net has been trained and tested using three optimizers, ADAM optimizer, Nesterov-accelerated Adam (NADAM) optimizer, and Stochastic gradient descent (SGD) for selecting the best features used to classify beehives abnormalities, Varroa destructor, hive-being robbed ant problems, and losing the queen. The most obvious finding to emerge from this study is the efficiency of the mobile net using ADAM optimizer in classifying beehives according to five classes of beehive health abnormalities. The proposed model achieved a testing accuracy of 95% and a testing loss of 35%. Compared to other deep learning approaches such as Shuffle Net, Resnet 50, VGG-19, and Google Net using the same optimizers. The evidence from this study suggests that Mobile Net can be used as an efficient deep learning methodology to recognize beehive health abnormalities. The study was limited to investigating beehive health abnormalities based on only beehive images, However, it is important to investigate the impact of bee sounds in recognizing beehive health abnormalities, this is what has to be investigated in future work.