ResNet18 Supported Inspection of Tuberculosis in Chest Radiographs With Integrated Deep, LBP, and DWT Features

The lung is a vital organ in human physiology and disease in lung causes various health issues. The acute disease in lung is a medical emergency and hence several methods are developed and implemented to detect the lung abnormality. Tuberculosis (TB) is one of the common lung disease and premature diagnosis and treatment is necessary to cure the disease with appropriate medication. Clinical level assessment of TB is commonly performed with chest radiographs (X-ray) and the recorded images are then examined to identify TB and its harshness. This research proposes a TB detection framework using integrated optimal deep and handcrafted features. The different stages of this work include (i) X-ray collection and processing, (ii) Pre-trained Deep-Learning (PDL) scheme-based feature mining, (iii) Feature extraction with Local Binary Pattern (LBP) and Discrete Wavelet Transform (DWT), (iv) Feature optimization with Firefly-Algorithm, (v) Feature ranking and serial concatenation, and (vi) Classification by means of a 5-fold cross confirmation. The result of this study validates that, the ResNet18 scheme helps to achieve a better accuracy with SoftMax (95

L UNGS are one of the chief internal organs in human physiology and infectious and acute disease in the lung will always lead to a medical emergency. Even though several preventive actions are taken, the occurrence rate of the infectious disease is steadily increasing due to different causes and appropriate recognition and treatment will help in reducing the impact of the syndrome and its spread.
An infection in the lungs will cause a severe respiratory problemand untreated lung disease causes increased morbidity and mortality. Tuberculosis (TB) is a communicable disease in the lungs, which causes a severe diagnostic burden globally. The occurrence rate of TB in a person is closely associated with the immune system and a weaker immune system will increase the chance of having TB [1] - [4].
TB is a communicable illness, which is initiated by a bacteria named Mycobacterium-Tuberculosis and the bacteria will spread through the air from an infected person. The discussion by Zaman [5] points to TB as one of the earliest diseases in mankind and along with the lung, it will affect sensitive organs, such as the brain, kidneys, spine, and intestines. TB easily affects people having a lower immune system and the happening rate of TB is progressively raising in low-and middleincome countries and causes a severe diagnostic burden. The report of World-Health-Organization (WHO) lists TB among one of the top 10 causes of death and hence several awareness and vaccination programs are regularly conducted to reduce TB. This report points out that, around 10 million global populations are infected with TB and this infection is small in children (12%) compared to adults and the elderly. Further, TB is found more in men (56%) compared to women (32%) and appropriate prevention and medication are necessary to control the occurrence rate and spread of TB [6].
The infection level of TB in a person is primarily tested using; (i) TB skin test and (ii) TB blood test [7], [8]. When the above tests confirm the risk of TB, the person is then diagnosed with bio-medical imaging procedures, such as Computed-Tomography (CT) scan and Chest Radiograph (X-ray) scan. Chest X-ray is widely preferred in detecting TB infection compared to CT because of its simplicity, reputation, and cost. The recorded X-ray slices are then inspected by an experienced pulmonologist and based on the detected infection score; the doctor will plan for necessary medication/treatment to cure the patient. In recent years computerized diagnostic procedures are widely employed to support doctors in detection, decision making, and treatment planning processes. Further, the employment of Artificial Intelligence (AI) and Deep-Learning (DL) schemes helped in increasing the diagnostic accuracy and speed of computerized methods employed in hospitals [9]- [11].
The recent literature confirms the implementation of pre-trained/ customary Deep-Learning-Procedure (DLP) to detect disease in various bio-medical image modalities [12]- [14]. Pre-trained schemes are widely employed due to their simplicity, readiness, and adaptability and a number of X-ray detection with pre-trained DLP confirmed its merit in accurately detecting the disease with appropriate clinical significance.
This research aims to develop a TB detection framework to inspect X-ray slices with improved accuracy. The proposed scheme consists of the following phases: (i) Image collection and initial processing, (ii) Employment of DLP to mine the deep features, (iii) Image enhancement and feature extraction using weighted Local Binary Pattern (LBP) and Discrete Wavelet Transform (DWT), (iv) Feature optimization using Firefly-Algorithm (FA), (v) Feature ranking and serial feature concatenation and (vi) Binary classification and validation.
This framework considered pre-trained DLP, such as AlexNet, VGG16, VGG19, ResNet18, ResNet50 and ResNet101 for the demonstration [15]- [18]. An experimental investigation is performed using X-ray images considered in the research work of Rahman et al. [19] and this experimental investigation confirms that ResNet18 with SoftMax helped to get better TB detection accuracy (95.2%) with deep features. Further, the ResNet18 scheme executed with a Decision Tree (DT) classifier helped to achieve a TB detection accuracy of 99% with integrated deep, LBP and DWT features. This research confirms that the pre-trained ResNet18 along with the proposed features helped to acquire an improved TB screening on the adopted image datasets.
The main contributions of this research include: (i) Assessment of pre-trained DLP on chosen image database (ii) Firefly algorithm supported feature optimization (iii) Validating the significance of the proposed technique with similar existing works.
The remaining sections of this study are structured as follows: Section II presents the motivation behind this research, Sections III and IV present the context and methodology. Section V presents experimental results and discussions, and the conclusion of our research is presented in Section VI.

II. Motivation
In the literature, a considerable amount of TB screening measures is presented using pre-trained/customary DLP and every method helps to get an appropriate result.
The proposed research is motivated by the recent works of Rahman et al. [19] and Rajakumar et al. [20] who invented a pre-trained DLP to examine the TB in X-ray images and attained considerable accuracy. Particularly, the work of Rahman et al. [19] presented two schemes; (i) Without segmentation and (ii) With segmentation and the overall detection performance of the TB detection framework is claimed to be superior compared to alternatives.
Employment of Convolutional Neural Network (CNN) segmentation and classification is a challenging as well as computationally complex task even though the pre-trained CNN schemes are adopted. The recent literature claims that the integration of the deep and handcrafted features helps to achieve better disease detection irrespective of the image modalities [21].
Hence, in this research, a Disease Detection Framework (DDF) is proposed to improve TB detection accuracy using pre-trained DLP with integrated features.

III. Related Work
The development of clinically significant DDF is necessary for accurate recognition and analysis of TB from X-ray images. In the literature, a number of DDF are presented using various AI and DLP techniques to recognize TB.
TB detection with a chosen AI/DLP involves in; clinical grade image collection and processing, DDF implementation for disease diagnosis, sharing the findings to the pulmonologist for decision making, treatment implementation and curing the patient. In this work, DDF development and implementation play a major role in automatic TB detection systems and the findings of this scheme can also be preserved to create disease models to improve the TB diagnosis process. Some chosen earlier DDFs proposed to identify the TB in X-ray are summarized in Table I. Earlier works in the literature implemented customary/pre-trained DLP to inspect the X-ray imagery related to TB using Montgomery dataset [23], Shenzhen dataset [23] and IEEE-Dataport [26] and every method discussed its merit on the considered database. The work of Kadry et al. [28] implements VGG-UNet scheme and achieves a detection accuracy of 99.22%. This approach is complex and time consuming compared to the proposed work, since it implemented segmentation and the classification task. The proposed research initially implemented the pre-trained DLP and tested its performance using SoftMax with a 5-fold cross-validation and confirmed that the ResNet18 provides better accuracy on the considered database.
After confirming the performance with deep features, the handcrafted features like LBP and DWT are then extracted from the test images and are then reduced using the FA. The reduced deep, LBP and DWT features are then successively concatenated, and the obtained feature vector is then considered to test the performance of the proposed DDF with SoftMax and other binary classifiers found in the literature [30]- [33].
The experimental outcome of this research confirms that ResNet18 with DT offers better TB detection compared to other methods.

IV. Methodology
Computerized disease screening is widely preferred in hospitals to decrease the investigative burden of doctors [34]- [36]. In most cases, computerized approaches are operated by skilled workers under the supervision of the doctors and the result (report) generated by the computer algorithm is shared with the doctor along with the patient information [37]- [39]. The doctor verifies the report and plans the necessary treatment procedure for the patient.
This work proposes a DLP-based DDF to distinguish TB in X-ray images and to improve TB detection accuracy, the image enhancement methods, such as LBP and DWT are adopted and integrated with the DLP. Fig. 1 depicts the basic framework proposed to examine the X-ray images of normal and TB class using the pre-trained ResNet18 scheme. Implementation of the pre-trained DLP-based TB detection is implemented using Python Ⓡ software and other works, such as feature extraction (LBP and DWT) and FA-based optimization are implemented in Matlab Ⓡ and the outcomes are then evaluated in Python's workspace. In this work, the required test images of Normal/TB class are collected from the benchmark datasets, and then every image is resized to 230x230x3 pixels. After the resizing, the test image database is separated into two sections, forming training and validation sets. The training set is initially considered to train the pre-trained models with a chosen image dimension (227x227x3 pixels for AlexNet and 224x224x3 pixels for the remaining DLP).
After the training, the performance of DDF is then verified using the validation images with a 5-fold cross-validation process and the necessary metrics, such as Accuracy (ACC), Misclassification (MIC), Precision (PRE), Sensitivity (SEN), Specificity (SPE), F1-Score (F1S) and Negative Predictive Value (NPV). Initially, the performance of the proposed scheme is tested using SoftMax and then, other well-known classifiers, such as Naïve-Bayes (NB), Linear-Discriminant analysis (LD), Decision-Tree (DT), Random-Forest (RF), K-Nearest Neighbor (KNN) and linear kernel Support-Vector-Machine (SVM) are considered.
This procedure is initially implemented using only the deep features and then repeated using FA optimized Deep, LBP and DWT feature set and the results are presented and discussed.

A. Image Database
The necessary test images for this research are acquired from the dataset by Rahman et al. [19] found in [26]. It consists of 7000 images (3500 TB and 3500 normal) of high clinical grade.
In this work, the test images which are having cropped sections are ignored and only 5000 images (2500 TB and 2500 normal) alone are considered for experimental investigation.
Before using the test images, every image is resized into 230x230x3 pixels, in which 90% (2250 images) images are considered for training the DLP and 10% (250 images) are utilized for validation. Further, the combined training and validation images are considered to test the performance of this scheme with 5-fold cross validation to prevent overfitting. Some sample test images of the dataset considered in this work are presented in Fig. 2.

B. Deep-Learning Procedure
Implementation of pre-trained/customary DLP is widely adopted to examine the clinical images recorded with a range of modalities and the DLP developed for a particular modality will work well on other modalities when it is trained with the new database of interest.
Developing and implementing a customary DLP is computationally complex, and it needs a considerable number of images to test and validate the performance. Hence, pre-trained DLPs are widely adopted to screen the medical images recorded with varied modalities. The conventional pre-trained DLPs are trained with the ImageNet database and a transfer-learning methodology helps to implement these schemes towards the new medical dataset which is to be examined.
In the proposed research, the pre-trained schemes, such as AlexNet, VGG16, VGG19, ResNet18, ResNet50 and ResNet101 are considered to categorize the considered X-ray image dataset into Normal/TB class. Initially, the performance of these DLPs is tested using SoftMax classifiers and the attained outcome confirms that the ResNet18 helped to get a better accuracy compared to other methods. Table II presents the information regarding the DLPs and their network parameters considered in this study. To implement a fair assessment, every network is assigned with similar parameters and the classification accuracy alone is adopted as the performance evaluation metric. In every image case, data augmentation is employed using; horizontal flip, vertical flip, rotation=20º, zoom=0.2, width shift=0.2, height shift=0.2, and shear range=0.1.
These images help the DLP to understand the information in the test pictures appropriately.
The ResNet18 has a simple structure compared to its variants and in the proposed work, this scheme helped to get better classification accuracy and therefore this architecture was considered to detect TB using deep and integrated deep, LBP and DWT features.

C. Handcrafted Features
The performance of automatic medical image examination is monitored by the features extracted from the test images considered for the assessment.
These features represent the technical information existing in the images and play a vital role in automatic segmentation and classification tasks in machine learning as well as DLP. The choice of a particular feature extraction procedure depends on; (i) Image modality, (ii) Methodology and (iii) Insight obtained from earlier literature. The previous works in the literature verify the use of LBP and DWT features to examine a class of images [40], [28], [29].

Weighted Local Binary Pattern
If LBP is an approved technique to mine the pixel level information from grayscale images and every LBP-treated image will help to get a one-dimensional feature vector of size 1x1x59. Recent procedures considered the LBP technique to mine the features from the normal/ disease class images and in this work, the LBP of varied weights (W= 1 to 4) discussed in Gudigar et al. [40] is considered. For the normal class X-ray images, approximately similar results are obtained. Other information related to the LBP features can be found in [16]. The LBP features of this study can be found below: (1) Eqns.
(1) to (4) present the features extracted for W=1 to 4 respectively and Eqn. (5) presents the arithmetic sum of total features extracted from all four images

Discrete Wavelet Transform
The concept of DWT-based feature extraction is one of the most widely adopted techniques and in this method, the chosen test image (RGB/grayscale) is divided into 4 sections, such as approximate-(LL), vertical-(LH), horizontal-(HL) and diagonal-coefficients (HH) and from every picture, necessary features are extracted as discussed in the work of Mirniaharikandehei et al. [41].
This work pointed out the essential 13 features to be extracted from the DWT treated images (contrast, correlation, energy, homogeneity, mean, standard-deviation, entropy, root-mean-square-level, variance, smoothness, kurtosis, skewness, and inverse-difference-moment) as discussed in the earlier research.
Eqns. (6) to (9) present the features extracted in LL, LH, HL and HH respectively, and Eqn. (10) presents the arithmetic sum of total features.

D. Firefly Algorithm-Based Feature Optimization
Feature reduction with a preferred statistical/heuristic scheme is commonly adopted in automatic data assessment to reduce the over-fitting problem and implementation of conventional statistical procedure (Student's t-test) is commonly adopted in the literature to reduce the features in medical data assessment [15]. Along with the statistical method, heuristic algorithm-supported feature selection is also widely considered by researchers to select the optimal feature set.
In this work, the Brownian-walk Firefly Algorithm (BFA) is employed to select optimal values of deep, LBP and DWT features and after the selection, serial integration of features is employed to form a new 1D feature vector of reduced size.
BFA is an improved form of FA, in which the search process is monitored by a slow and steady Brownian-walk operator [42]. When this scheme is implemented, the agents (artificial fireflies) are allowed to slowly explore the normal/TB class features and select the features whose Hamming-Distance (HD) is large. In this work, the initial parameters for the BFA are allocated as; number of agents=30, search operator=Brownian motion, monitoring parameter=maximization of HD and number of iterations (Iter max ) = 1000. The search will be stopped after finding the necessary feature vector from the chosen features and in this work, the optimized values of the features of each case are presented in Eqns. (11) to (13).
Eqns. (11) to (13) present the BFA selected features of Deep, LBP and DWT and all these features are then serially combined to get a new feature vector as presented in Eqn. (14). This feature vector is then adopted to train and confirm the binary classifiers.

E. Performance Validation
The merit of the proposed TB detection framework is tested and authorized using the well-known binary classifiers, such as SoftMax, NB, LD, DT, RF, KNN ad SVM and the necessary information about these techniques can be obtained from [17]- [20]. During the performance evaluation, the necessary metrics, such as ACC, MIC, PRE, SEN, SPE, F1S and NPV are computed and based on these values; the merit of the proposed DDF is verified.

V. Experimental Results and Discussion
This part of the work reveals the investigational result achieved by the proposed DDF and this scheme is implemented using a workstation of; Intel i5 2.5 GHz CPU with 16GB RAM and 4GB VRAM and equipped with Python®.
In this work, 2500 images of Normal/TB class images are considered for the assessment, in which 90% of images (2250) are considered for the training and 10% images (250) are used for the validation process. During the deep-feature supported classification with a 5-fold crossvalidation, SoftMax based binary-classification is implemented and the attained performance metrics are evaluated.
The experimental outcome confirms that ResNet18 combined with the SoftMax helps to achieve a classification accuracy of >95%, which is better compared to other DLP adopted in this study. Due to its performance, the ResNet18 scheme is then considered to construct the necessary DDF to examine the X-ray pictures.
ResNet18 consists of four separate levels as presented in Fig. 1 and each layer learns based on the information available in the images. The learned pixel information obtained from all these four levels is presented in Fig. 7, in which Fig. 7(a) to (d) represents the outcomes of levels 1 to 4, respectively. The final level result is then connected to the fully connected layer, which then presents a 1D learned (deep) features with size 1×1×1000. This feature is initially employed to train and authorize the SoftMax classifier existing in the DLP, and the attained results are presented in Fig. 8 and Table III  In this work, a 5-fold cross-validation is considered, and the best result is selected as presented in Table IV. In this table, Fold5 presents the best accuracy compared to other folds. Table III and IV values are graphically compared as presented in Fig. 9 (spider-plot) . Fig 9(a) confirms the merit of ResNet18 and Fig. 9(b) verifies the merit of considered validation (Fold5).   The achieved result of this experiment confirms that the deep feature-based classification helped to get an accuracy >on95% with SoftMax. To verify other classifiers' merit, this experiment is repeated using the deep features and the attained results are then compared and in Table V and Fig. 10. Fig 10(a) verifies that the binary classification with the DT helps to get better accuracy (>96%) on the considered X-ray picture dataset with deep feature.
In order to improve the accuracy further, the integrated deep, LBP and DWT features (Eqn. (14)) are considered, and the experiment is repeated with various classifiers using 5-fold cross-validation. The outcome of this experiment is also presented in Table V and Fig 10(b) verifies that the classification with DT helps to get a better result (99%) compared to other approaches.
The Glyph-plot of Fig 10(b) confirms that the texture formed by DT is big compared to the alternatives, which verifies the overall merit of the DT classifier. This experimental investigation confirms that the ResNet18 with DT classifier helps to get better TB detection using the X-ray images. Further, the proposed DDF is tested using deep-feature and integrated features and the outcome verifies that the integrated feature supported DDF works well compared to the alternatives.
This research work confirms that the proposed scheme helps to get a TB detection accuracy of 99%. To verify the merit of this scheme, the results of Rahman et al. [19] and Rajakumar et al. [20] are compared with the outcome of the proposed scheme along with other methods discussed in Table I Kadry et al. [29] Rohilla et al. [28] Rajakumar et al. [24] Rahman et al. [23] Hooda et al. [11] Hijazi et al. [10] Afzali et al. [1] Fig. 11. Performance evaluation of the result of proposed work with recent existing works.    11 presents the evaluation of the proposed scheme with the existing scheme and this comparison confirms that the classification accuracy of the proposed technique is better compared to the earlier works. Further, these results outperform the results of dual-deep features of Rajakumar et al. [20] and CNN segmentation combined classification of Rahman et al. [19] on the chosen image database. In the future, the performance of the REsNet18 supported DDF can be tested and validated on other existing TB datasets and clinically collected X-ray images.

VI. Conclusion
Tuberculosis (TB) causes severe lung problems and early screening, and treatment will reduce the impact. X-ray-supported TB detection is a common clinical scheme, and the recorded X-ray is then examined by radiologists and pulmonologists to detect the infection rate and plan the treatment.
Computerized X-ray detection is very common in hospitals and to support computerized screening of TB, this work proposed a DDF using the pre-trained DLP. This work initially executes a performance assessment of the DLPs such as AlexNet, VGG16, VGG19, ResNet18, ResNet50 and ResNet101 existing in the literature and finds that the ResNet18 along with SoftMax classifier helps to get a better classification result.
To improve the result of this scheme, the deep features are then optimized with FA and then combined with FA-optimized LBP and DWT features. This scheme helps to get a reduced 1D feature vector of dimension 1×1×544, which is then considered to train and validate the classifiers.
The DT classifier with a 5-fold cross-validation helped to get a classification accuracy of 99% with this feature vector and it is close to the earlier work implemented with CNN segmentation and classification. This outcome substantiates that the developed DDF helps to obtain an improved TB screening compared to the earlier methods. In the future, this scheme can be considered to evaluate the other X-ray image dataset in the literature and clinically collected X-ray images.

V. Rajinikanth
He is a Professor in Department of Computer Science, Division of Research and Innovation, Saveetha School of Engineering, SIMATS, Chennai 602105, Tamilnadu, India. He has published more than 150 papers and authored/edited 8 books in the field of medical data assessment. His main research interests includes; Heuristic algorithm based optimization, Image thresholding, Machine learning and Deep learning.

Seifedine Kadry
Professor Seifedine Kadry has a Bachelor degree in 1999 from Lebanese University, MS degree in 2002 from Reims University (France) and EPFL (Lausanne), PhD in 2007 from Blaise Pascal University (France), HDR degree in 2017 from Rouen University. At present his research focuses on Data Science, education using technology, system prognostics, stochastic systems, and applied mathematics. He is an ABET program evaluator for computing, and ABET program evaluator for Engineering Tech. He is a Fellow of IET, Fellow of IETE, and Fellow of IACSIT. He is a distinguished speaker of IEEE Computer Society.

Pablo Moreno Ger
Dr. Moreno-Ger was born in Madrid in 1981. He finished his doctorate in Computer Engineering from Universidad Complutense de Madrid (UCM) in 2007 and was an Associate Professor in the Department of Software Engineering and Artificial Intelligence at UCM. Now he is with Universidad Internacional de La Rioja (UNIR), where he is currently the Vice-Rector for Research. Formerly, he was the Director of the School of Engineering and Technology at UNIR, as well as Vice-Dean for Innovation at the School of Computer Engineering at UCM. His main research interests are in technology-assisted teaching, artificial intelligence, learning analytics and serious games. He has published more than 150 academic works in these fields.