Classification of Covid-19 patients using efficient fine-tuned deep learning DenseNet model

As COVID-19 pandemic caused completely spoils the livings, almost more than one year passed, still lives were not on the track. It is important to diagnose the COVID-19 patients earlier and provide the prompt treatment. The Convolutional Neural Network (CNN), a deep neural network that specializes in image processing and image classification. In this paper, a fine tuned DenseNet201 model was proposed which is used to classify Chest X ray images. Firstly, different DenseNet121, DenseNet169 and DenseNet201 model trained and tested on the same dataset. With the experiment, it is observed that DenseNet201 model performs well as compared to other dense models. Furthermore, DenseNet201 experiments over different optimizers and it is noticed that RMSprop, Adagrad and Adamax performs better. Proposed model achieves accuracy of 95.2% as compared to other models. We experimentally determine that RMSprop optimizer with DenseNet201 produces better results as similar to Adam and Adamax widely used optimizers.


Introduction
In an unforeseen situation, our civilization is yet again fighting a novel coronavirus, SARS-CoV-2 (Severe Acute Respiratory Syndrome Coronavirus-2), in another historical battle. According to a study, the virus originated in Wuhan, China, around December 2019. As Soon after, the coronavirus was dubbed COVID-19, and due to it communicable properties spreads quickly across the world, prompting the World Health Organization (WHO) to proclaim a global pandemic in March 2020 [1] .
However, given the significance of the human community, it has become necessary to develop an autonomous approach that can diagnose COVID-19 in a short amount of time. A crucial key initiative is accurate and effective COVID-19 patient detection, so that positive cases undergo prompt treatment and are properly segregated from the general population; a precaution believed critical in preventing the spread of the disease [ 2 , 3 ]. In recent years, AI techniques have been widely used in a variety of challenges ranging from classification, segmentation, and face identification to upgrade-resolution and image enrichment in computer vision and medical image analysis [4] . As COVID-19 is a respiratory disease which directly harms the lungs and cause difficulty in breathing. For the diagnosis of COVID patient's chest X-ray images plays a vital role. For the accurate and fast results these images can be examined using the Deep learning techniques which is already proves its excellence in the field of image processing and analysis [ 5 , 6 ].
In this paper, a novel approach is presented for the detection of COVID-19 images using deep learning DenseNet201 model. The suggested DenseNet model modification ensures flow of information by directly connecting all layers in the network to their feature maps. The feedforward method is maintained by obtaining additional values from prior layers and passing feature maps from the previous layers on to all subsequent levels [7] . Using the TensorFlow Python library, the suggested model was tested in practice and yielded encouraging results for analysis of the data. The main contribution of this can be summarized as follows: • Provide a literature survey, authors using DenseNet model for the classification of X-ray images. As none of paper explains the different DenseNet models and claims which optimizer provide the best results. The rest of the paper is structured as follows: Section 2 represents the related work; Section 3 defines the concepts of DenseNet model with different number of layers. Further, Section 4 describes the different optimizers used in the article and section 5 represents the proposed work followed by result analysis done in Section 6 . Last but not the least conclusion is summarized in Section 7 .

Related work
Computer vision assists us in developing autonomous systems that do tasks that are similar to and, in some circumstances, better than human vision. One of the most important contributions of computer vision is in the field of medicine improved illness diagnosis, therapy, and prevention using data on medical imaging [8] . A deep neural network excellently performs in the field of image processing and Convolutional Neural Networks (CNN) widely used for this purpose. As there are various forms of CNN such as Resnet, VGG, Xception and many more [9] . This section represents the deep learning techniques used for the classification of COVID chest X-ray images. Moreover, abundant of research work done on CNN model used for COVID-19 detection but this paper concentrate only on the DenseNet CNN model, hence we explored only those author's work proposed by various researchers and experts where DenseNet model used in their research works.
In this section, we focus on deep learning DenseNet model which also a type of CNN model. Further, we summarize some related and existing work where DenseNet model used to classify the COVID-19 chest X-ray images. In the paper [10] , author experiments the various deep learning models and DenseNet121, DenseNet169 and DenseNet201. Total 746 images were used by the author for the training and testing of the models. With the experiments, author shows DenseNet201 performs excellently as compared to other deep learning models. In the paper [11] , author proposed a CNN model and compared with pretrained deep learning model. Author used two data sets of 111 images and 6290 chest X-ray images. Author also used DenseNet201 for the comparisons. Though in his experiments, DenseNet201 doesn't perform well enough.
In the article [12] , author proposed a fused DenseNet model which is a concatenation of two DenseNet model. Furthermore, author used fused DenseNet for the classification of 9208 chest X-ray images and achieved accuracy of 97.99%. In the article [13] , author runs various deep learning models for the classification of covid19 patients. Author also includes DenseNet121 model for the experiment and total 5000 images were used to train and test the model.
In the article [14] , author proposed a model by fine tuning the pretrained DenseNet201 model and further used for the classification of 950 chest X-ray images. In the paper [15] , author used DenseNet CNN model to classify the 2482 CT scan images. Preprocessed and augmented dataset used for training of the model.
In the paper [16] , author proposed DenseNet-OTLS for the effective detection of COVID-19. All the three DenseNet121, DenseNet169 and DenseNet201 were used for the experiment and total 640 images were used to train and test the model. Author worked on two factor single learning rate and Composite learning rate for the optimization of DenseNet model.
In the article [17] , author proposed a CovidDenseNet model using DenseNet121 for the diagnosis of COVID-19. Author experiments both two and three class classification and patient's wise k-fold was also performed more than 13,800 chest radiography images were used for the experimentation. Further, Grad-CAM used to highlight the infected regions of lungs for better clarity in reading images.
With the study of all the articles mentioned above DenseNet CNN model performs well as compare to other CNN models but there is a gap which DenseNet model performs well under which circumstances and furthermore we try to explore the working of different optimizers with this research article.

Deep learning CNN model
Deep learning [18] focuses on several levels of abstraction, with higher layers representing more abstract data knowledge. As an alternative to traditional machine learning techniques for identifying COVID-19 chest X-ray images, neural networks can be used. Every layer of the CNN in DenseNet is interconnected to every other layer in the network in a feed-forward way, which reduces the chance of gradient vanishing, reduces the number of parameters to train, reuses feature maps, and each layer takes all previous layer features as inputs [ 19 , 20 ]. Table 1 explains the dense layers for the different DenseNet models. Mainly, DenseNet model consist of many dense blocks and one dense block contains convolutional layer, ReLU layer and batch normalization. On the other hand, two dense blocks connected with convolutional and max pooling layer and the last dense block connected with global average pooling and softmax classifier [21] .

Optimizers
Optimization Algorithms, also known as Optimizers, are essential for improving the accuracy of a neural network. In a traditional approach, hyper parameters of a model were tuned using optimizers according to their design. An optimizer's job is to manipulate the weights and learning rates of our model's nodes during the training phase so that the loss function is effectively minimized [ 22 , 23 ].

Stochastic Gradient Descent or SGD
Stochastic Gradient Descent or SGD is a version of Gradient (or slope of a function) Descent, which is the simplest basic optimization algorithm. Although too simple to be employed in Deep Learning, the latter has a wide range of applications in Linear Regression, Classification, Backpropagation, and other areas, with the benefit of ease of computation and implementation. Stochastic (means random) Gradient Descent, on the other hand, chooses a few sample data rather than the complete dataset for each round, resulting in a significant increase in estimating speed [24][25][26] .

Root mean square propagation or RMSprop
RMSprop is a gradient-based optimizer that uses an adaptive learning rate that changes over time rather than considering the learning rate as a hyper-parameter. RMSprop, or Root Mean Square Propagation, has an unusual fact related with it: despite its popularity, it is an optimizer that has never been released. In his online course on Neural Networks for Machine Learning, Geoff Hinton, the father of backpropagation, proposed it.

Adaptive Gradient Algorithm (Adagrad)
Although the Adaptive Gradient Technique (Adagrad) is quite similar to the stochastic gradient descent algorithm, show that it does not use adaptive gradients to improve robustness. One of Adagrad's key advantages is that it eliminates the need for manual learning rate calibration, whereas it's most serious fault is the aggregation of squared gradients in the denominator.

Adadelta
AdaDelta is a modified successor to Adagrad and another part of the family of stochastic gradient descent algorithms. The default learning rate has been removed from the update rule, thus there is no need to change its default value. It also offers adaptive hyper-parameter tweaking approaches and is visibly resistant to chaotic gradient details. By providing a fixed size, it imposes constraints on the aggregate previous gradients [27][28][29] .

Adam or Adaptive Moment Estimation
The most prevalent optimizer is Adaptive Moment Estimation, or Adam. It has a lot in common with RMSprop and Adagrad. Adam employs the L2 norm, often known as the Euclidean norm, for optimization. It is known for its great efficiency, adaptability, and speedier convergence [30] .

Adamax
Adamax and Adam were both introduced in the same paper. It's a variation of the Adam optimizer that uses infinity or max norm for optimization. Adamax will outperform Adam with data that is conventionally unstable in terms of gradient updates (for example, a dataset with several misfits) [ 31 , 32 ].

Proposed methodology
In this article, author proposed a new model using the pre-trained DenseNet201 model. This study chose to employ a densely connected neural network, the DenseNet model, for the proposed model, as shown in Figure 2 Fig. 2 . The goal of using a DenseNet model is to ensure that features propagate flawlessly across the network without efficiency depletion, even as the depth is increased. Let summarized the whole process of proposed model in further paragraphs.

COVID-19 dataset
The proposed model evaluated on Covid data set collected from Kaggle repository. The data set consist of chest X-ray images categorized into 3 classes Covid, normal and viral pneumonia and further this data set divided into train and test folders for the experimentation work. The train data set containing 111 images of Covid,70 images of normal and 70 images of viral pneumonia. For the validation 20% of data from train folder taken. The test data set contains 26 images of Covid, and 20 images of normal and 20 images of viral pneumonia. Fig. 3 Figure 3 explains the sample images from the data set it is clearly observed the normal image of chest X-ray contains the white opacity surrounded by the lungs were as this opacity reduces in case of viral pneumonia and this opacity almost zero for Covid chest X-ray images.

Transfer learning and fine tuning for proposed model
The main principle behind transfer learning is that the skills learned when studying a model can be applied to a different learning assignment. CNNs are based on a series of deeper layers, with input passing through a number of them. The input data may be lost before it reaches the network's final layer.

COVID-19 detection using DenseNet model
Each layer in DenseNet receives extra inputs from all preceding levels and sends its own feature-maps to all subsequent layers. Each layer receives a collective knowledge from the levels above it. Because each layer receives feature maps from all previous layers, the network can be

Model training and hyper-parameters
Hyper-parameters are the parameters which plays an important role in training the model and values selected for the hyper-parameters largely affects the model accuracy. Most commonly known hyperparameters for the deep learning model are learning rate, batch-size, number of epochs, number of dense layer and weights. The hyperparameter defines the proposed model were mentioned in the Table 2 .
The overall design of the proposed fine-tuned DenseNet201 model defined in the Figure 2 Fig. 2 . DenseNet201 model is used for the proposed methodology as we already performed experiment with DenseNet121, DenseNet169 and DenseNet201 on the same data set. For the proposed methodology we used the concept of data augmentation on the chest X-ray images. After data augmentation images were ready for the input to the model and different optimisers SGD, RMSprop, adagrad, adadelta, adam and adamax with the same learning rate. All the top layers containing the dense blocks were freeze as these layers already trained on the ImageNet database. Further, dense layer with the activation function at output layer softmax is added for the classification of data set containing three classes normal, Covid and viral pneumonia. We trained are model by using the hyper-parameters already defined in Table 2 . Complete model runs for 10 epochs with the batch size equal to 3.

Metrics for model evaluation
To evaluate categorization predictions, four types of metrics were computed.

True Positive (TP)
label prediction done by model for an input image and image actually belongs to the same label i.e. an Chest X-ray that is classified as COVID and is actually belongs to COVID Class.

True Negative (TN)
label prediction made by model for an input image and image actually does not belong to that class i.e. model predicted an image does not belong to NORMAL class and that image actually not NORMAL.

False Positive (FP)
image classified as COVID by the model but actually not belongs to COVID class.

False Negative (FN)
image classified as not COVID but actually belongs to COVID class. These four outcomes were explained with the help of confusion matrix and better defined the results of the proposed model. Moreover, Accuracy, Precision, Recall and F1-score are the four vital metrics used for the classification and Eqn 2 and 3 Eqs. (1) -(4) explain all the four metrics.  F1 score is the harmonic mean of precision and recall.

Results analysis
In this section, we discuss the various computed results. The first part of our experimentation is the exploration of different DenseNet models.  Figure 4,5,6,7,8,9,10,11,12,13,14 and 15 shows the training and validation accuracy and loss against the different optimizers. Table 4 represents the performance of all the optimizers used in the experimentation work whereas Figure 10 Figure 18 fig. 18 is the confusion matrix for the proposed methodology. Table 5 compares the proposed methodology with the existing art of work.

Conclusion
In this paper, a fine tuned DenseNet model proposed for the classification of COVID-19 patients. DenseNet121, DenseNet169 and DenseNet201 were experimented on the same dataset and with the same hyper-parameters values and RMSprop optimizer. DenseNet201 provides better results as compared to the other DenseNet models. DenseNet201 tuned and using RMSprop optimizers performs well. For future work, more research can be done on the DenseNet201 and optimization of these models for accurate and effective classification of COVID-19 using Deep learning models. While the results are intriguing, more research on a larger sample of COVID-19 photos is needed to provide a more credible estimate of accuracy rates.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.