Health status detection of neonates using infrared thermography and deep convolutional neural networks
Graphical abstract
Introduction
The temperature value of the body is vitally important for health and has been used in medical applications since 400 BC [1]. Healthy skin is characterized by thermal symmetry [2], whereas diseases and infections cause local temperature changes resulting in consequently occurring thermal asymmetry on the skin surface. Hence, temperature measurement devices are used to identify any thermal changes and provide information that facilitates the detection and diagnosis of diseases. In conventional methods, the temperature value is obtained via sensors and electrodes. In all aforementioned materials, the absolute zero point (0 K) emits infrared radiation, and thermal cameras convert the infrared radiation into electrical signals [3], [4]. These electrical signals are then converted into thermograms, developed by a processing unit, that include both temperature values and thermal representation. Thermograms have been widely used in various environmental [5], industrial [6], and medical [7] studies. Thermal imaging is called Infrared Thermography (IRT) in medicine. IRT, which is a non-invasive and non-contact method, has been used in medical studies in the fields of thermoregulation [8], breast cancer detection [9], [10], neonatal monitoring [11], urology [12], [13], and vascular diseases [14], [15].
With the development of machine learning algorithms, the application of automatic analyzes such as anomaly detection and lesion segmentation instead of conventional template operations has considerably increased. Therefore, feature engineering methods, including feature extraction [16] and feature selection [17] are becoming increasingly important. Object detection [18], pattern recognition [19], [20], medical image classification [21], [22], and image segmentation [23], [24] have been effectively implemented.
The late diagnosis of diseases and anomalies increases mortality rates. According to statistics of the World Bank [25], the mortality rate in neonates was 18 in 2017. Pre-diagnosis and follow-up treatment systems can be designed by observing the temperature distribution of the skin, and the obtained information can be provided to medical teams. Frequently, the temperature detection and analysis in neonates is performed by electrodes and sensors attached to their sensitive skin [26]. In comparison to the use of sensors and electrodes, IRT provides more capabilities, such as monitoring of the heart and respiratory rates, as well as sleeplessness and restlessness.
The first study on neonatal IRT was conducted by Clark and Stothers in 1980. They observed the skin temperature distribution of neonates using thermography and a thermocouple thermometer [11]. A mean square error of 0.107 was obtained in that investigation. In 2012, Abbas et al. proposed compensation techniques for different clinical scenarios, such as convective incubators, kangaroo care, and open radiant warmers [27]. Then, in 2013, Ruqia performed abdominal thermal symmetry analysis for early diagnosis of neonates with necrotizing enterocolitis (NEC) disease [28]. The results of this earlier study showed a higher degree of thermal asymmetry in the group with NEC than in the normal group.
Later, in 2014, Abbas and Leonhardt reported the preliminary results of their feature analysis and Newborn Infrared Thermography (NIRT) imaging [29]. The authors performed decomposition of NIRT images based on independent component analysis and computed the first- and second-order statistical parameters. They also proposed intelligent neonatal monitoring [30]. Skin temperatures of neonates were recorded using thermography, and a vector-based active follow-up system was designed by identifying the region of interest. In 2017, Knobel et al. measured the abdominal and foot temperature of neonates using skin thermistors and thermography [31]. In 2018, Savasci and Ceylan presented the first evaluation results of thermal image analysis for NICU [32]. The obtained results showed differences between the thermal symmetry degrees of healthy and unhealthy neonates. Additionally, Ornek et al. conducted thermal approach analysis using wavelet transform with a thermal map and RGB images [33]. They demonstrated the importance of using a thermal map instead of RGB images.
In recent decades deep learning models, such as multilayer perceptron, CNNs, and recurrent neural networks have been rapidly developing. CNN models [34] were found to have high performance in resolving visible [35], [36], [37], [38], [39], [40], [41] and thermal [42], [43], [44], [45], [46], [47], [48] image-based problems, such as segmentation, classification, and detection.
Despite a large number of studies on deep learning and thermal imaging, no report related to neonates is available in the literature. Therefore, to the best of our knowledge, this is the first study on the detection of the health status of neonates as healthy and unhealthy by IRT and CNNs.
The rest of the paper is organized as follows. In Section 2, measurement setup and data are presented. In Section 3, methods such as networks architecture, data augmentation, evaluation metrics, and cross-validation are described. Section 4present the detailed results and the subsequent section describes the conclusion.
Section snippets
Measurement setup and data
Neonatal thermal images were taken in Selcuk University, Faculty of medicine, NICU over a one-year period. The measurement setup implemented is illustrated in Fig. 1.
Thermal images were obtained using IRBIS, designed by Infratec© Vario- cam HD infrared camera, which is a product of Infratec, was used for the recording of neonatal thermal images. The temperature resolution of the thermal camera is up to 0.02 K at 30 Celcius and the measurement accuracy ±1 Celcius or ±1%, the resolution
Methods
In the succeeding subsections, we describe the data augmentation for the thermal images, the architecture of the proposed CNN model, the evaluation metrics, and cross-validation. The block diagram of the proposed system is illustrated in Fig. 3. Data augmentation methods were implemented using MATLAB. The development of the CNNs model and the classification were accomplished by PYTHON and Keras library which uses TensorFlow backend. All the process was run on Nvidia Quadro K2200 4 GB 128 Bit
Results
In this study 380, 3800, 15,200, and 30,400 thermal images belonged to 38 neonates that were classified as healthy and unhealthy. Confusion matrices were calculated using a 10 – fold cross-validation method. Sensitivity, specificity and accuracy metrics were obtained from confusion matrices and these metrics were used to evaluate the performance of classification. The obtained ROC curve is depicted in Fig. 7 and all results are presented in Table 4.
We used 380 thermal images by taking 10
Discussion and conclusions
Thermal imaging which is a non-contact and non-invasive method provides more capabilities than visible imaging. For example, temperature values of the body can be obtained with thermal imaging even in darkness. Thermal imaging applications are called Infrared Thermography (IRT) in medicine. Body temperature is considerably important for health status evaluations, and since diseases, disorders and infections cause thermal anomalies over the body, IRT can be used to detect breast cancer and
Declaration of Competing Interest
The authors declared that there is no conflict of interest.
Acknowledgments
This study was supported by the Scientific and Technological Research Council of Turkey (TUBITAK, project number: 215E019). The authors express their gratitude to Selcuk University’s expert pediatricians H. Soylu and M. Konak, for their help and future vision. We also thank all the staff who helped during the process of taking thermal images of the neonates in the neonatal intensive care unit.
References (57)
- et al.
Medical applications of infrared thermography: a review
Infrared Phys. Technol.
(2012) - et al.
Infrared thermography – a non-invasive tool to evaluate thermal status of neonatal pigs based on surface temperature
Animal
(2013) - et al.
Effectiveness of a noninvasive digital infrared thermal imaging system in the detection of breast cancer
Am. J. Surg.
(2008) - et al.
Digital infrared thermal imaging (DITI) of breast lesions: sensitivity and specificity of detection of primary breast cancers
Clin. Radiol.
(2011) - et al.
Pattern recognition using artificial neural networks
Comput. Chem. Eng.
(1992) - et al.
An information fusion based method for liver classification using texture analysis of ultrasound images
Inform. Fusion
(2014) - et al.
Neonatal infrared thermography imaging: analysis of heat flux during different clinical scenarios
Infrared Phys. Technol.
(2012) - et al.
Body temperature in premature infants during the first week of life: exploration using infrared thermal imaging
J. Therm. Biol.
(2017) The generalized sigmoid activation function: competitive supervised learning
Inf. Sci.
(1997)- R. Dail, B. Guenther, H.E. Rice, Thermoregulation and Thermography in Neonatal Physiology and Disease,...
Applying dynamic thermography in the diagnosis of breast cancer
IEEE Eng. Med. Biol. Mag.
Temperature tracking system for sinter material in a rotatory cooler based on infrared thermography
IEEE Trans. Ind. Appl.
Neonatal skin temperature distribution using infra-red colour thermography
J. Physiol.
Transurethral Needle Ablation (TUNA): thermal gradient mapping and comparison of lesion size in a tissue model and in patients with benign prostatic hyperplasia
Eur. Urol.
”Investigation of peripheral vascular disorders using thermal imaging
Br. J. Diabetes Vasc. Dis.
Infrared thermal imaging for detection of peripheral vascular disorders
J. Med. Phys.
Multiresolution gray-scale and rotation invariant texture classification with local binary patterns
IEEE Trans. Pattern Anal. Mach. Intell.
A novel approach for the classification of liver MR images using complex orthogonal ripplet-II and wavelet-based transforms, classification in BioApps: automation of decision making
Cited by (25)
Artificial intelligence in neonatal and pediatric intensive care units
2023, Artificial Intelligence in Clinical Practice: How AI Technologies Impact Medical Research and ClinicsFast Evaluation of Unhealthy and Healthy Neonates Using Hyperspectral Features on 700-850 Nm Wavelengths, ROI Extraction, and 3D-CNN
2022, IRBMCitation Excerpt :In this way, better results were achieved by obtaining different features in different spectrums. In addition, 2D data was obtained as a result of thermal imaging in [35] and the classification process was performed using 2D-CNN. Here they only acquired spatial features.
Detecting abusive Instagram comments in Turkish using convolutional Neural network and machine learning methods
2021, Expert Systems with ApplicationsCitation Excerpt :Convolution operation represents the extraction of the features of the text depending on the selected kernel. Pooling operation refers to reducing in the dimensions of the obtained feature map according to pooling size and selected methods such as average and maximum pooling (Ornek, Ceylan, & Ervural, 2019). We have employed a CNN model as an ANN-based classifier for abusive message detection.
Cancer detection in breast cells using a hybrid method based on deep complex neural network and data mining
2023, Journal of Cancer Research and Clinical OncologyCloud environment-based super resolution application for thermal images using the new approach TSRGAN+ model
2023, Multimedia Tools and Applications