Pneumonia Detection using Depth-Wise Convolutional Neural Network (DW-CNN)

around 15 % of the total deaths of children of the same age group. OBJECTIVES: This paper proposes Depth Wise Convolution Neural Network (DW-CNN) using the SWISH Activation and Transfer Learning (VGG16) to reliably diagnose pneumonia. METHODS: The proposed model contains 10 layers of convolutional neural networks. Also, three dense layers with the Swish activation function with a dropout of 0.7 and 0.5 respectively in each layer. The model was trained on 5216 augmented with weighted contrast and brightened radiograph Images and tested on 624 radiogram images using Deep Learning and Transfer Learning (VGG16). RESULT: The model was trained on 5216 augmented radiograph Images and tested on 624 radiogram images using Deep Learning and Transfer Learning (VGG16) and the final results obtained with training accuracy of 98.5%, testing accuracy of 79.8% and validation accuracy of 75%. CONCLUSION


Introduction
Pneumonia is a type of disease that occurs in lungs.Bacteria, viruses, and fungi are the cause of it.The disease causes inflammation in the air sacs(alveoli) in the lungs.The alveoli are filled with pus making it difficult to breathe, and causing coughing that may produce phlegm, sweating, fever, and also shortness of breath.There are high chances of infants, people aged 65 and above, and people with some health problems or weak immunity systems to get infected with the disease.The largest deaths among children due to disease is caused by Pneumonia worldwide.It led to the death of around 808,694 children having age less than 5 years in 2017, which constituted about 15% of the total deaths of children of the same age group [1].Many computer aided tools have been invented to analyze the radiographic image of the chest [2].Delay in diagnosis and adequate treatment are considered to be the major reasons accounting for the high rate of pneumonia among young age people.Therefore, these particular cases motivated INTRODUCTION: Pneumonia is most significant disease in today's world.It resulted around 15 % of the total deaths of children of the same age group.OBJECTIVES: This paper proposes Depth Wise Convolution Neural Network (DW-CNN) using the SWISH Activation and Transfer Learning (VGG16) to reliably diagnose pneumonia.METHODS: The proposed model contains 10 layers of convolutional neural networks.Also, three dense layers with the Swish activation function with a dropout of 0.7 and 0.5 respectively in each layer.The model was trained on 5216 augmented with weighted contrast and brightened radiograph Images and tested on 624 radiogram images using Deep Learning and Transfer Learning (VGG16).RESULT: The model was trained on 5216 augmented radiograph Images and tested on 624 radiogram images using Deep Learning and Transfer Learning (VGG16) and the final results obtained with training accuracy of 98.5%, testing accuracy of 79.8% and validation accuracy of 75%.CONCLUSION: The model can be improved by using different transfer learning models and hyperparameter tuning parameters.

Inderpreet Singh Walia et al.
us for performing a study on the pneumonia detection by leveraging deep learning technologies [3].Supine Chest radiograph (SCXR) is generally the primary imaging technique done in intensive care patients who are suspected of pneumonia due to its low costs and radiation dose as well as its availability on bedside.Diagnosis of SCXR regarding pneumonia is considered difficult.This especially concerns the basal lung zones, where maximum diseases occur [4].In recent years, Convolution Neural Network has emerged as a standard solution for medical image analysis.CNN possesses an optimized structure in order to handle 2D and 3D shapes and also have the potential to extract 2D features through learning [5].Now with the help of Computer Vision several tools are developed to provide a perception of radiograph images.The information gathered from Deep Learning and Computer Vision has significantly helped the doctors in making decisions swiftly [6].A normal chest radiograph image taken from the dataset [7] has been shown in Figure 1.The organization of the is as follows: Section 2 explains the literature done in this area to detect pneumonia using different techniques and algorithms.Section 3 defines the methodology of the model that has been used throughout the paper to detect pneumonia and also give the details of the dataset on which the model has been tested.It also describes the architecture of the proposed model.Section 4 discusses the results and analysis of the proposed methodology.The outcomes of proposed methodology have been compared with the existing algorithms followed by conclusion.

Related Work
There have been numerous researches based on chest, lungs and tuberculosis diseases using machine learning [8].Artificial Neural Networks (ANN) have been used to explore chest and lung diseases.In a study, it was observed that chest diseases could be diagnosed successfully using ANN.Best results were obtained using PNN for chest diagnosis problems [9].A structured analysis of distinct approaches for CNN classification on Chest radiograph images have been done recently [10].In another research paper, inspection and identification of chronic pneumonia was performed by ANN [11].
In another study, a CNN model was proposed which could accurately detect pneumonia from chest radiograph samples and it also deployed data augmentation techniques in order to improve the accuracy of the model [12][13].As per study, a model was trained using a dataset which consisted of 112,120 images collected from 30,805 unique patients [14].The model was trained to help doctors in diagnosing pneumonia in chest radiograms [15].Different procedures are present in literature that can help to recognize pneumonia by working on chest radiography images.In another study different deep CNN models were proposed to extract features from images of chest radiography [16].
Latest studies show that deep learning has emerged as a powerful tool in medical image analysis, bioinformatics, object detection, segmentation and natural language processing, etc. [17][18][19].Research has shown that diagnosis accuracy is based on factors like patient poisoning, quality of image and the variations in radiologist threshold for diagnosis [20].Chest radiographs with CAD systems improve accuracy in desirable ways.In a research, deep CNN has gained a leading position because of its performance in classification of images and object recognition for natural images in ImageNet [21].In another research, the author has proposed their work on CAD methods to distinguish between bacteria and virus pneumonia via Chest radiographs.They first extracted the lung region using FCN model and DCNN method by using the dataset from public JSRT Database [22] and MC dataset [23] The authors have proposed a multi-layered capsule called capsNet to reorganize pneumonia from chest radiograph images.An ICC & ECC (Integration & Ensemble of convolutions) has been proposed to detect pneumonia [24].Rajpurkar et al. [25] proposed a new 121 sheets CNN which classified pneumonia among 14 other diseases based on Chest X rays, called cheXNet.
Improved accuracy and sensitivity in extracting features by DCNN were observed in the results [26].Radiograms are effective to detect chest diseases but many times it remains a difficult task resulting in fatigue-based diagnosis error [27].In a study, it was observed that lungs having pneumonia absorb more radiation and the affected area appears white on Chest radiogram resulting in understanding the severity of disease [28].Research has proposed a deep learning-based model that distinguishes chest radiograph as normal or pneumonia.Using the dataset of 5856 radiograph images, the model has the classification accuracy of 94.39% and high sensitivity (0.99) which is very promising [29][30].According to a study, the isolation rate observed from pneumonic cases was 22% (195 out of 885) while that of non-pneumonic cases was 11% (66 out of 660) [31].The statistics show that the isolation rate for the pneumonic cases outnumbered those of the non-pneumonic ones.
Pneumonia is a major cause of morbidity among the children worldwide.The World Health Organization (WHO) has estimated that acute respiratory diseases are the cause of deaths of around 4 to 5 million children per year [32].The most common symptom among the patients was cough, about 66% [33].

EAI Endorsed Transactions on Pervasive Health and Technology
Online First

Materials and Methods
The proposed model aims to determine pneumonia accurately in chest radiography images using Depth-Wise CNN.The algorithms consist of 10 layers of CNN.After employing the processing methods, augmenter modules have been implemented so as to overcome the imbalances in the dataset of images.To reduce overfitting, an ordinance technique (dropout) has been employed at the dense layers.
The model consists of 10 layers of CNNs and it was trained using ReLu activation function which introduces linearity.Also, three dense layers with the swish activation function and dropout of 0.7 and 0.5 respectively have been used to determine pneumonia in chest radiography images.
We undertook the detailed experimental and evaluation measures for checking the efficiency.The proposed model uses the chest radiograph data for experimentation.The open source deep learning framework, Keras was deployed in backend tensorflow for constructing the model consisting of CNN [34].All of the experimentation was executed on a system with a configuration of AMD radeon RX 590 GPU card of 6GB, cuDNN v7.2 library, and CUDA Toolkit 10.0.

Dataset
The dataset is categorized into 3 folders (i.e.train, test and validation) and each folder is divided into subfolders for each category of the image (Pneumonia/Normal).There are overall 5,863 radiograph images (JPEG) which are divided into 2 categories (Pneumonia and Normal).
In the Guangzhou Women and Children's Medical Center, Guangzhou the retroactive part of the under-five pediatric patients was taken in consideration for the chest radiograph images which were both anterior and posterior.During the clinical health checkup of a patient, the process of imaging all the chest radiographs were performed.
All chest radiographs were originally scanned for quality assurance and all tedious scans were removed for the analysis of chest radiograph images.Before the AI system was trained, the identification of images was done followed by evaluation of the images by two expert physicians.For accounting the evaluation set of classify errors was checked by a third expert in order to remove any discrepancies.The training set was allocated an amount of a set of 3722 images while the validation set consisted of 2134 images to increase the validation accuracy.

Preprocessing and Data Augmentation
The size and quality of data were enhanced via preprocessing methods.The problems like overfitting were solved via this process and the ability of generalization was improved in the process of training.To overcome imbalances in the datasets of images, an augmenter module had been implemented.The number of image files in the dataset had been artificially increased by using transformations which didn't change the classes, the images belonged to.Each image had been transformed according to the steps mentioned below: Step

Dropouts
To reduce overfitting, a technique that brings the uniformity called dropout has been used at the dense layers.Dropout has been set to 0.7 for the first dense layer(fc1) and the activation function used is swish and the dropout for second dense layer(fc2) has been set to 0.5 and the same activation is used as before.This ensures that the neurons in convolutional and ANN layers are mostly independent of each other, in a single layer.

Model Architecture
Depth Wise Convolution neural network-based approach  The architecture model of CNN is a simple model that uses data augmentation.It has its first convolution layer (i.e.Conv1_1) as an input with the kernel size (3,3), along with relu activation (as per equation ( 1)) it is an activation function that introduces uniformity.The equation for the ReLu activation formula has been given below in equation (1).
ReLu = max (0, x), for x>=0 Further, a dropout of 0.7 has been introduced after this layer (in dropout1).These are then flattened (in flatten) to get 100352 values which are passed through a fully connected layer of 1024 neurons (in fc1).The last dropout of 0.5 has been introduced in the model which has been passed through a fully connected or dense layer(fc2) to get output using swish activation function.The equation for swish activation function has been mathematically described in equation (2).

𝑦𝑦(𝑥𝑥) =
where i = 1, 2…, n and x = (x1, x2…, xi) The sigmoid function is kept at the other end of the proposed CNN model and is used as a binary function to give the binary results.The output of layers is altered into 1D feature vector form using the approach of flattening for the dense layers to perform the overall specified classification process.This classifying layer consists of a fully connected layer with a dropout of 0.5 and also consists of dense layers of sizes 64 and 2 respectively.The sigmoid activation function performs the process of classification.The optimizer used is ADAM with a learning rate of 0.009 and loss is binary_crossentropy.

EAI Endorsed Transactions on Pervasive Health and Technology
Online First dataset of 5216 augmented with weighted contrast and brightened radiograph Images, the images are validated on 16 images and tested on 624 radiograph images.A thorough collation of these models has been done in Table 1 while their training curves has been illustrated in Figure 4 (a -e) and Figure 5 2), ( 3), ( 4) and ( 5) for the categorization of radiograph images.1), ( 2), ( 3), ( 4) and ( 5) for the categorization of radiograph images.
Methods like data processing, learning rate variation, padding, dilation rate and pooling were employed, because the data was invariant.The reason for the invariance in the data is that there are thrice images of pneumonia (bacterial and viral) as compared to a normal lung image in the dataset.The described model makes an effective use of data augmentation and gives desirable results by reducing the False Negatives which are wrongly classified instances.In the practical world there is no importance to the false negatives.The confusion matrix for the proposed model has been shown in Figure 7.It is being discovered that VGG16 has made a tremendous use of data augmentation.The confusion matrix results with recall accuracy of 99% and precision accuracy with 70% which illustrate that the model is susceptible to overfitting when it has to train on an aforesaid dataset.
Deep learning-based approach using data augmentation and simple CNNs have been used to detect pneumonia.The algorithms consist of CNNs, different transfer learning methods like VGG16, VGG19 and Inception v1 and the results of which have been depicted in Table 1.
The analysis given in table 1 suggests that most of the models that were trained had been subjected to overfitting of some kind except those that used data augmentation (i.e."DA used" column).But the models that had used data augmentation have given a low accuracy.Hence, an inference that can be derived from the table is that the models with data augmentation suffer from low accuracy while they avoid overfitting.Examples of this can be seen in the case of simple CNN, VGG16 or VGG19 which obtain an accuracy of 72% and with data augmentation 78%, 80% and 86%, while the precision of 53% and with data augmentation 66%, 68%, 71% whereas recall of 70% and with data augmentation 77%, 80%, 87% respectively.On the other hand, we can observe that the INCEPTION v1 transfer learned model has achieved an accuracy of 90%, precision of 67% and recall of 93%.While working with our model that is DW-CNN performs with the accuracy of 95 % precision of 70% and recall of 99%.The bar graph comparison of all the models with various evaluation criteria has been shown in Figure 8.

Discussions
We came out with a model to identify and classify pneumonia from the dataset containing chest radiograph images, from the retrospective groups of pediatric patients at high validation accuracy.

Conclusion and Future Scope
The model that has been introduced here, using Depth-Wise Convolution Neural Network (DW-CNN) and transfer learning (VGG16) obtained with a training loss of 2.1% which is binary cross-entropy and training accuracy of 98.5% when trained on a set of 5216 augmented radiograph Images.The model is prone to overfitting and obtained a validation accuracy of 75% and validation loss of 42.9% where loss is binary cross-entropy.The model has been tested for 624 radiograph Images and obtained a correctness of 72.43%, loss of 58.5%.Also, the model has been trained for 2 hours and 30 minutes.The model uses an optimizer known as Adam to enhance the accuracy of the model and 3 activation functions namely RELU, SWISH, and SOFTMAX [35].When a problem contains an imbalance data, then it is not good to use accuracy metrics in such cases.
For example, dataset containing 5 negative and 95 positives, having a model with 95% accuracy does not mean anything.
The model may classify every example as negative and still get 95% accuracy.Hence there is a need to look alternative metrics for such problems, therefore, Precision and Recall are good metrics for such problems.Multiple techniques like batch-normalization, Max-pooling in the model which causes loss in data.
This model can be expanded further by making use of different algorithms and techniques.By using different transfer learning algorithms and freezing the transfer learning algorithm layers, this model can be improved, and hyperparameter tuning will further improve this model's characteristics.This can be expanded to identify various types of diseases using different datasets.

Figure 1 .
Figure 1.The normal chest radiograph depicts clear lungs in the image.

Figure 2 .
Figure 2. Dataset distribution 1. imgaug had been used for image augmentation in CNN and to open the files in the class folders and saved as PNG files.1.a.Rotation: Rotating an image to 20°.1.b.Horizontal Flip: An image flip reverses the rows pixels in the image array.1.c.Random Brightness: Values greater than 1 brighten up the image, we used [1.2,1.5] for brightening up the image.

Figure 3 .
Figure 3.The proposed architecture

( 2 )
Finally, the output has been passed through the final fully connected layer (i.e, fc3) by using a softmax activation function (in equation (3))
The described model has been trained on 5216 augmented & augmented with weighted contrast and brightened radiograph Images, validated on 16 images and tested on 624 radiograph images.Figure6 (a-b) represents the training accuracy and training loss as well as validation accuracy and loss for the proposed model.

Figure 6 (Figure 6
Figure 6(a): Training Accuracy (in green) and Validation Accuracy (in blue) during the training of model for the categorization of radiograph images;

Figure 7 .
Figure 7. Confusion Matrix for the model The model was initiated by reducing the size of chest radiograph images as compared to the original image.The next course of action involved the identification and categorization of images by using Depthwise Convolution Neural Network (DW-CNN) and Transfer Learning (VGG16), which extracted the feature respectively.To corroborate the working of the trained model on numerous images of chest radiographs of different sizes, the dimensions of the training and validation dataset were varied, still we obtained relatively homogenous results.With increase in access to data, specifically radiological data obtained from the patients can aid in detecting pneumonia more accurately.

Table 1 .
Comparative Analysis of different CNN models EAI Endorsed Transactions on Pervasive Health and Technology Online First .Figure 8. Performance Comparison of Deep Learning Models