SKIN LESION CLASSIFICATION FROM DERMOSCOPY AND CLINICAL IMAGES WITH A DEEP LEARNING APPROACH

Computer-aided diagnosis (CAD) systems based on deep learning approaches are now feasible due to the availability of big data and the availability of powerful computational resources.The medical image-based CAD systems are of great interest in numerous diseases, but especially for skin cancer diagnosis, deep learning models have been mostly developed for dermoscopy images. Models for clinical images are few, mainly due to the unavailability of big volumes of relevant data. However, CAD systems able to classify skin lesions from clinical images would be of great valueboth for the population and clinicians as an initial early screening of lesions that would leadpatients to visiting a dermatologist in case of suspicious lesions. This is even more pronounced in areas where there is lack of dermoscopy instruments. Thus, in this paper, we aimed to build a classifier based on bothdermoscopy and clinical images able to discriminate skin cancer from skin lesions. The classification is made among three benign and two malignant categories, which include Nevus, Benign but not nevus, Benign but suspicious for malignancy, Melanoma and Non-Melanocytic Carcinoma.The proposed deep learning classifier achieves an Area Under Curve ranging between 0.75 and 0.9 for the five examined categories.

Computer-aided diagnosis (CAD) systems based on deep learning approaches are now feasible due to the availability of big data and the availability of powerful computational resources.The medical imagebased CAD systems are of great interest in numerous diseases, but especially for skin cancer diagnosis, deep learning models have been mostly developed for dermoscopy images. Models for clinical images are few, mainly due to the unavailability of big volumes of relevant data. However, CAD systems able to classify skin lesions from clinical images would be of great valueboth for the population and clinicians as an initial early screening of lesions that would leadpatients to visiting a dermatologist in case of suspicious lesions. This is even more pronounced in areas where there is lack of dermoscopy instruments. Thus, in this paper, we aimed to build a classifier based on bothdermoscopy and clinical images able to discriminate skin cancer from skin lesions. The classification is made among three benign and two malignant categories, which include Nevus, Benign but not nevus, Benign but suspicious for malignancy, Melanoma and Non-Melanocytic Carcinoma.The proposed deep learning classifier achieves an Area Under Curve ranging between 0.75 and 0.9 for the five examined categories.

…………………………………………………………………………………………………….... Introduction:-
Skin cancer is increasingly evident worldwide (Goyal et al., 2020) with the most common forms of skin cancer being non-melanocytic, such as Basal Cell Carcinoma (BCC) and Squamous Cell Carcinoma (SCC). Cutaneous melanoma, on the other side, although being rare, has been associated with high mortality rates (Rebecca et al., 2020). Early detection of skin cancers is of crutial importance in an effort to reduce mortality. Thus, researchers have shown interest in building automated frameworks to aid skin cancer diagnosis, starting from 1985 when the ABCD rule (Ali et al., 2020) was devised. More sophisticated computer aided diagnosis (CAD) systems using dermoscopy or clinical images started subsequently to emerge (Barata et al., 2018). The use ofmachine learning and deep learning algorithms in CAD systems made them even more powerful, but numerous limitations are still present (Goyal et al., 2020) and have to be faced before such systems start to be part of the clinical practice.
Medical image-based CAD systems for skin cancer diagnosis with deep learning approaches have been mostly developed for dermoscopy images. Models for clinical images are few, mainly due to the unavailability of big ISSN: 2320-5407 Int. J. Adv. Res. 9(10), 1294-1300 1295 volumes of relevant data. This poses limitations to deep learning methods, because they depend on data availability, since they learn from the data in order to be able to perform predictions.However, CAD systems able to classify skin lesions from clinical images would be of great value both for the population and clinicians as an initial early screening of lesions that would lead patients to visiting a dermatologist in case of suspicious lesions. This is even more pronounced in areas where there is lack of dermoscopy instruments.
In the present paper, we build a classifier based on both dermoscopy and clinical images able to discriminate skin cancer from skin lesions. Dermoscopy and clinical images for building and testing the classifier were retrieved from ebioMelDB (Korfiati et al., 2021), a collection of skin cancers multimodal data and PAD-UFES-20 (Pacheco et al., 2020),a recently published dataset which contains clinical images.The classification is made among three benign and two malignant categories, which include Nevus, Benign but not nevus, Benign but suspicious for malignancy, Melanoma and Non-Melanocytic Carcinoma. Performance metrics are measured in a validation set of images. The proposed deep learning classifier achieves an Area Under Curve ranging between 0.75 and 0.9 for the five examined categories in the validation set.Melanoma is predicted with an AUC of 0.78, while non-melanocytic carcinomas with an AUC of 0.9.

Data classes
Maintaining the categorization of the images suggested in ebioMelDB our data are organized into five categories: NEV (Nevus), MEL (Melanoma), NNV (Benign non-nevus), NMC (Non-melanocytic carcinoma) and SUS (Benign but suspicious for malignancy). The benign categories are NEV, NNV and SUS. The nevus category NEV includes simple, common, blue, clark, combined, con-genital, dermal, recurrent, reed or spitz nevus. The benign non-nevus category NNV includes dermatofibroma, lentigo, melanosis, miscellaneous, seborrheic keratosis, vascular lesion, benign keratosis, cafe-au-lait macule, lentigo NOS and lichenoid keratosis. The benign, but suspicious for malignancy category SUS includes actinic keratosis, atypical melanocytic proliferation and atypical nevus. The malignant melanoma category MEL includes melanoma and melanoma metastases images. Finally, the category of non-melanocytic carcinomas includes basal cell carcinoma and squamous cell carcinoma.

Model description
Learning good representations, along with a deep enough deep learning architecture, requests the underlying training data to be massive. Training from scratch such a model can be deterrent in certain domains due to the lack of labeled data. In order to minimize the negative impact of data scarcity, we applied a network-based transfer learning (Tan et al.,2018) and fine tuning along with data augmentation. The proposed model for lesion classification into the five classes is presented in Figure 1.

Training
In order to train the model, we used heavy augmentation techniques on the available images. The underrepresented classes on each task were oversampled considering also the image type of each sample. We used cross entropy objective function and AdaMax(Kingmaet al.,2014) optimization algorithm with initial learning rate = 10 −5 , 1297 fine-tuned the best performing model by applying weight updates to the pretrained model with an initial = 10 −6 and applying the same early stopping strategy.

Setup
We implemented the proposed method with python 3.8 and Tensorflow framework (Abadi et al.,2016). The data augmentation was implemented with tensorflowkeras image preprocessing and tensorflow-addons image preprocessing modules. For the input pipeline we used the tensorflow dataset API. This work was supported by computational time granted from the National Infrastructures for Research and Technology S.A. (GRNET S.A.) in the National HPC facility -ARIS -under project ID pa210303. The development environment was on Redhat/Centos 6.7 OS and the provided hardware of each GPU node used was 2x Haswell -Intel(R) Xeon(R) E5-2660v3 CPU, 64 GByte RAM and 2x NVIDIA Tesla K40 GPU.

Results:-
In order to evaluate the performance of the proposed model we run it with the following parameters.
Learning  In the validation set we computed the precision, recall and F1 score metrics as shown below and the receiver operating characteristic -ROC area under the curve -AUC.
The results for these metrics are presented in Table 2  The respective ROC curves for each class are also presented in Figure 3. In the following matrix, we can observe how images from the five actual different classes were classified in the model's predictions.

Discussion:-
The proposed multiclass classification scheme gives quite encouraging results in the classification of skin lesions and the detection of skin cancers. The benign class nevus is predicted correctly with a precision of 0.89 and a recall of 0.61. The second-best classification performance is achieved for the malignant class non melanocytic carcinoma followed by melanoma as the third best predicted class. This can be attributed to two main reasons. The remaining two benign categories include benign lesions with a) high diversity and at the same time b) few images for each lesion type. Thus, having not enough data for some classes leads to the class imbalance problem which does not allow deep learning algorithms to perform well.

1299
Additionally, from the results it can be observed that benign lesions of one of the three categories are misclassified as one of the other benign lesions categories. As a future direction, we plan to treat the problem as a binary problem trying to discriminate skin cancers from benign lesions. Another interesting observation is that melanoma is mostly misclassified as nevus and vice versa. Again, we also plan to create a model able to distinguish melanoma from nevus. Another interesting future direction would be the integration of patient clinical data together with the clinical and dermoscopy images.
Improvement of the classification performance could also be achieved with improvements in the deep learning model. Dealing with the class imbalance problem, experimenting with different data augmentation techniques and preprocessing of the images in order to remove hair and other marks are potential paths for a better classification model.

Conclusion:-
Computer-aided diagnosis systems for skin cancer have been of great interest for a number of years, but only recently with the availability of big volumes of data and of powerful computational resources, deep learning approaches were enabled to emerge in skin cancer related studies. Such approaches have been mostly developed for dermoscopy images because the unavailability of big volumes of clinical images does not ease the development of clinical images based CAD systems. However, these would be of great value both for the population and clinicians as an initial early screening of lesions alarming patients to visiting a dermatologist in case of suspicious lesions. This is even more pronounced in areas where there is lack of dermoscopy instruments.
Herein, we described a deep learning model based on both dermoscopy and clinical images able to discriminate skin cancer from skin lesions. The classification is made among three benign and two malignant categories, which include Nevus, Benign but not nevus, Benign but suspicious for malignancy, Melanoma and Non-Melanocytic Carcinoma. The proposed deep learning classifier achieved encouraging results giving at the same time numerous future directions for improvement.