Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

The early detection of skin cancer, specifically, the so called melanoma is one of the main issues analyzed by medical oncologists and dermatologists as the probability of full oncological remission is strongly correlated to early detection. A robust and efficient approach for nevus discrimination is currently under investigation by physicians as well by bio-medical engineers with the target to develop a non-invasive “Point of Care (PoC)” for real-time skin cancer detection. Currently, both oncologists and dermatologists use heuristics approach to analyze skin lesion known with acronyms of “ABCDE” which means “Asymmetry, Border irregularity, Color variegation, Diameter, Evolution” applied to a classical dermoscopy image. Basically, the ABCDE criteria performs visual statistical assessment of the nevus based on physician experience and background. Clearly, this strategy suffers from clinician subjectivity as low sensibility and specificity; often it is required a nevus invasive biopsy to confirm diagnosis. In order to address that issue, the authors propose an automatic skin lesion pipeline based on analysis of dermoscopy images in order to discriminate benign lesions from malignant ones trying to have a good trade-off between sensibility and specificity. The method has been successfully validated by using dermoscopy image dataset of PH2 open database [3]. Several approaches have been proposed in the past for utomatic and robust detection of the skin cancer, including statistical hand-crafted features or soft computing approaches based on machine learning algorithms. In [9] the authors review several recent methods for melanoma detection basically based on usage of image features, neuro-fuzzy approaches, clustering methods (K-means, SVM, etc.) and some methods based on the study of melanotocytes distribution on the skin histopathologic images as well as based on probabilistic analysis (e.g. Bayesian Classifier). The overall analysis reports some very promising methods based on the usage of neural networks for classifying skin lesions or based on adaptive thresholding analysis. Among other methods based on classical pattern analysis approaches combined with classical statistical dermoscopy image features pointed-out acceptable results [9]. In [9, 10] the authors shows a method for classifying skin cancer based on usage of global and local features combined with different classification systems such as SVM, ANN, K-nearest, Naive-Bayes Algorithm showing drawbacks and advantages of the analyzed approaches. An interesting approach is proposed in [11] where some hand-crafted image features are combined together with a deep-learning algorithm able to extract other ad-hoc features. A final SVM engine is then used for performing finale classification by means of a scores based approach.

The results reported in [11] shows a limited sensibility/specificity for the analyzed image dataset. Other approaches [12, 13] makes use of “standard” image features coupled with a classification engine (i.e., SVM or K-nearest, etc.) with limited results.

The proposed method is based on combined approach of ad-hoc customized hand-crafted image features combined with a feed-forward neural network system which properly trained, is be able to perform a robust and efficient classification system.

2 Proposed Pipeline: System Description

In Fig. 1 the proposed system is synthetically detailed. Starting from preliminary dermoscopy acquisition made by dermatologist, till to the output of the algorithm consisting of numerical array containing representation of morphologic features applied to the pre-processed dermoscopic image as well as a classification of the skin lesion in terms of probability of oncological malignant progression.

Fig. 1.
figure 1

The proposed feed-forward skin lesion analysis pipeline

Full size image

2.1 Dermatoscope and SC-CNNs Pre-processing Block

The dermoscopy image is basically a RGB colour image acquired by classical medical optical dermatoscope. The medical dermatoscope provides also a zooming option for enlarging the image of the analyzed skin lesion. In Fig. 2 we show a classical dermoscopy image of a nevus:

Fig. 2.
figure 2

A typical dermoscopy image of a skin nevus

Full size image

In our pipeline, we avoid to use direct colour information; the method is applied only to luminance component of the source RGB dermoscopy image “D(x, y)” after classical conversion into YCbCr format [2]. The luminance component denoted as “Y D(x,y) ” is fed as “input” and “state” to a State-Controlled Cellular Neural Networks (SC-CNNs) having 2D matrix structure and size equal to source image Y D(x,y) of size m × n.

The classical Cellular Neural (or Nonlinear) Network (CNN) was firstly introduced by Chua and Yang [14]. The CNN can be defined as high speed local interconnected computing array of analog processors [14]. The CNN processing is defined through the instructions provided by the so called cloning templates [14]. Each cell of the CNN array may be considered as dynamical system which is arranged into a topological structure, usually a 2D or 3D grid. The CNN cells interacts each other within its neighborhood defined by heuristically ad-hoc defined radius [14]. Each CNN cell has an input, a state and an output which is a functional mapping of the state (usually by means of PWL function). The CNN can be implemented with analog discrete components or VLSI technology so that it is able to perform high speed “near real-time” computations. Some stability results and consideration about the dynamics of the CNNs can be found in [14, 15]. An updated version of original Chua-Yang CNN model was introduced by Arena et al. in [15] and called “State Controlled Cellular Neural Network (SC-CNN)” as it explicates dependency of dynamic evolution of the cell to the “state” of the single cell directly. We refer to SC-CNNs in the following mathematical formulations. By assigning each normalized gray-level of the input source image (Y D(x,y) of size m × n) to each cell of the SC-CNNs (input and state of each cell), several image processing tasks may be performed according to the defined cloning templates instructions [14]. The state equation of a (m × n) SC-CNNs can be defined as follows:

$$ \begin{aligned} C\frac{{dx_{ij} (t)}}{dt} & = - x_{ij} (t)\text{ + }\sum\limits_{{C(k,l) \in N_{r} (i,j)}} {A(i,j;k,l)y_{kl} (t)} + \sum\limits_{{C(k,l) \in N_{r} (i,j)}} {B(i,j;k,l)u_{kl} (t)} \\ & + \sum\limits_{{C(k,l) \in N_{r} (i,j)}} {C(i,j;k,l)x_{kl} (t)} + I \\ & N_{r} (i,j) = \left\{ {C(k,l} \right.),(\hbox{max} (\left| {k - i} \right|,\left| {l - j} \right| < r\} \\ & k \in [1,m],l \in [1,n] \\ & y_{ij} (t) = 0.5\left( {\left| {x_{ij} (t) + 1} \right| - \left| {\left| {x_{ij} (t) - 1} \right|} \right|} \right) \\ \end{aligned} $$
(1)

In (1) the N r (i, j) represents the neighborhood of each cell C(i, j) with radius r. The terms “x ij ”, “y ij ”, “u ij ”, “A(i, j; k, l)”, “B(i, j; k, l)”, “C(i, j; k, l)”, “I” are respectively: the state, the output and the input of the cell C(i, j), the cloning templates (A, B, C) and the bias (I) i.e. an heuristic ad-hoc defined constant.

The proposed SC-CNNs block can be used for several pre-processing operations such as lesion segmentation, edge detection, noise reductions, pixels averaging and so on [15]. In this paper we propose the usage of SC-CNNs for skin lesion pre-processing with novel ad-hoc cloning templates setup. Anyway, robust segmentation of the nevus is another challenge tasks because there are several issues related to noise removal (hairs, angiomas, etc.) without significant distortion of the original ROI of the lesion image. In this work we perform validation of our method by using dermoscopy images provided by PH2 database in which segmented mask for each medical image is also provided [2] (Fig. 3).

Fig. 3.
figure 3

Pre-processing block based on SC-CNNs

Full size image

The SC-CNNs performs transient pre-processing of gray-level converting lesion image Y D(x,y) by using the following cloning templates [2, 8]:

$$ A = \left[ {\begin{array}{*{20}c} 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \\ \end{array} } \right] ; {\text{ B}} = \left[ {\begin{array}{*{20}c} 3 & {0.25} & {0.25} \\ {0.25} & 3 & {0.25} \\ 3 & {0.25} & {0.25} \\ \end{array} } \right] ; {\text{ C}} = \left[ {\begin{array}{*{20}c} 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \\ \end{array} } \right]; \, I = 0.7 $$
(2)

Above cloning templates is useful to configure SC-CNNs in order to perform ad-hoc adaptive time-transient increasing of gray-level Y D(x,y) image contrast reducing the image distortion produced by the typical gel used by dermatologist during medical dermoscopy. The dermoscopy gray level image processed by SC-CNNs will be denoted as Y SC-CNN(x,y) . The following figure shows an example of dermoscopy image pre-processed by SC-CNN based block as described in Fig. 3:

The original dermosocpy image is also segmented by using same algorithm and image-mask used in PH2 database as shown in Fig. 4.

Fig. 4.
figure 4

(a) Original RGB dermoscopy image (b) image-mask provided by PH2 database (c) SC-CNNs pre-processed segmented gray-level image.

Full size image

2.2 The Hand-Crafted Image Features and SC-CNNs Hand-Crafted Blocks

At this level, the pre-processed gray-level dermoscopy image named \( Y_{SC - CNN(x,y)}^{\prime } \) is further processed by using ad-hoc morphological-heuristic set of hand-crafted features useful to reproduce the widely used ABCDE rule [1]. The image features are re-scaled via logarithmic function in order to compress the ranging of their values. Some of them are classical statistical indicators while other ones are heuristically defined encoding in some sense the ABCDE rule, with high precision and robustness. Moreover, some of the proposed hand-crafted image features tries to analyze the melanocytes distribution on the skin epidermis [1]. We denotes with nr and mr the dimension of the bounding box enclosing the signal (Fig. 5).

Fig. 5.
figure 5

Some geometric parameters for the segmented SC-CNNs gray-level image.

Full size image

We indicate p(i, j) as gray-level intensity of the single image pixel while and \( \varTheta \left( {p^{\prime}\left( {i,j} \right)} \right) \) a measure of the frequency rate of the pixel \( p^{\prime}(i,\,j) \). In the following the overall list of employed feature F i is reported

$$ F_{1} = { \log }\left( {\frac{1}{m \cdot n}\mathop \sum \limits_{i = 1}^{mr} \mathop \sum \limits_{j = 1}^{nr} p\left( {i,j} \right)} \right) $$
(3)
$$ F_{2} = { \log }\left( {\frac{1}{mr \cdot nr}\mathop \sum \limits_{i = 1}^{m} \mathop \sum \limits_{j = 1}^{n} \left( {\left| {p\left( {i,j} \right) - F_{1} } \right|} \right)} \right) $$
(4)
$$ F_{3} = { \log }\left( {\frac{1}{mr \cdot nr}\mathop \sum \limits_{i = 1}^{m} \mathop \sum \limits_{j = 1}^{n} \left( {p\left( {i,j} \right) - F_{1} } \right)^{2} } \right) $$
(5)
$$ F_{4} = \sqrt {|F_{3} |} $$
(6)
$$ F_{5} = { \log }\left( {\frac{\pi }{2} \cdot \frac{1}{m \cdot n}\mathop \sum \limits_{i = 1}^{m} \mathop \sum \limits_{j = 1}^{n} \left( {\left| {p\left( {i,j} \right) - F_{1} } \right|} \right)} \right) $$
(7)
$$ F_{6} = - \mathop \sum \limits_{i = 1}^{m} \mathop \sum \limits_{j = 1}^{n} \left( {\varTheta \left( {p^{\prime } \left( {i,j} \right)} \right) \cdot log\left( {\varTheta \left( {p^{\prime (i,j)} } \right)} \right)} \right) $$
(8)
$$ F_{7} = { \log }\left( {\sqrt {\left| {F_{3} - (F_{6} )^{2} } \right|} } \right) $$
(9)
$$ F_{8} = { \log }\left( {\frac{1}{mr \cdot nr}\mathop \sum \limits_{i = 1}^{m} \mathop \sum \limits_{j = 1}^{n} \left( {\frac{{\left| {p\left( {i,j} \right) - F_{1} } \right|}}{{F_{4} }}} \right)^{3} } \right) $$
(10)
$$ F_{9} = { \log }\left( {\frac{1}{mr \cdot nr}\mathop \sum \limits_{i = 1}^{m} \mathop \sum \limits_{j = 1}^{n} \left( {\left| {i - \frac{m}{2}} \right|} \right) \cdot \left( {\left| {j - \frac{n}{2}} \right|} \right)\left( {\frac{{p\left( {i,j} \right) - F_{1} }}{{F_{4} }}} \right)^{4} } \right) $$
(11)
$$ F_{10} = { \log }\left( {\frac{1}{mr \cdot nr}\mathop \sum \limits_{i = 1}^{m} \mathop \sum \limits_{j = 1}^{n} \left( {\left| {i - \frac{m}{2}} \right|} \right) \cdot \left( {\left| {j - \frac{n}{2}} \right|} \right)\left( {\frac{{\left| {p\left( {i,j} \right) - F_{1} } \right|}}{{F_{4} }}} \right)^{5} } \right) $$
(12)
$$ F_{11} = { \log }\left( {\frac{1}{mr \cdot nr}\mathop \sum \limits_{i = 1}^{m} \mathop \sum \limits_{j = 1}^{n} \left( {\left| {i - \frac{m}{2}} \right|} \right) \cdot \left( {\left| {j - \frac{n}{2}} \right|} \right)\left( {\frac{{p\left( {i,j} \right) - F_{1} }}{{F_{4} }}} \right)^{6} } \right) $$
(13)
$$ F_{12} = { \log }\left( {\frac{1}{mr \cdot nr}\mathop \sum \limits_{i = 1}^{m} \mathop \sum \limits_{j = 1}^{n} \left( {\left| {i - \frac{m}{2}} \right| \cdot \left| {j - \frac{n}{2}} \right| \cdot p\left( {i,j} \right)} \right)} \right) $$
(14)
$$ F_{13} = { \log }\left( {\frac{1}{mr \cdot nr}\mathop \sum \limits_{i = 1}^{m} \mathop \sum \limits_{j = 1}^{n} \left( {\left| {i - \frac{m}{2}} \right| \cdot \left| {j - \frac{n}{2}} \right| \cdot \left( {p\left( {i,j} \right)} \right)^{2} } \right)} \right) $$
(15)
$$ F_{14} = { \log }\left( {\frac{1}{mr \cdot nr}\mathop \sum \limits_{i = 1}^{m} \mathop \sum \limits_{j = 1}^{n} \left( {\left| {i - \frac{m}{2}} \right| \cdot \left| {j - \frac{n}{2}} \right| \cdot \left( {p\left( {i,j} \right) \cdot \left( {i - j} \right)^{2} } \right)} \right)} \right) $$
(16)
$$ F_{15} = { \log }\left( {\frac{1}{mr \cdot nr}\mathop \sum \limits_{i = 1}^{m - k} \mathop \sum \limits_{j = 1}^{n - k} \left( {\left| {i - \frac{m}{2}} \right| \cdot \left| {j - \frac{n}{2}} \right| \cdot \left( {\left| {p\left( {i,j} \right) - p\left( {i + k,j + k} \right)} \right|} \right)} \right)} \right) $$
(17)
$$ F_{16} = { \log }\left( {\frac{1}{mr \cdot nr}\mathop \sum \limits_{i = 1}^{m - k} \mathop \sum \limits_{j = 1}^{n - k} \left( {\left| {i - \frac{m}{2}} \right| \cdot \left| {j - \frac{n}{2}} \right|\left( {\left( {\left| {p\left( {i,j} \right) - F_{1} } \right|} \right) \cdot \left( {\left| {p\left( {i + k,j + k} \right) + - F_{1} } \right|} \right)} \right)} \right)} \right) $$
(18)
$$ F_{17} = { \log }\left( {m \cdot n \cdot \frac{1}{6} \cdot \frac{1}{mr \cdot nr} \cdot \left( {\left( {F_{8} } \right)^{2} + \left( {\frac{1}{4} \cdot \left( {F_{9} - 3} \right)^{2} } \right)} \right)} \right) $$
(19)

Note the feature F 17 is a modified version of the “Jarque-Bera index” usually used in the field of financial markets able to point-out kurtosis and skewness of time-series. The authors adapted its meaning to their purpose related to 2D analysis of the kurtosis and skewness features for the analyzed nevus. In Eq. (17) we used k = 5 while in Eq. (18) we used k = 3.

$$ F_{18} = { \log }\left( \begin{aligned} \frac{1}{mr \cdot nr} \cdot \left( {\mathop \sum \limits_{i = 1}^{{round\left( {\frac{m}{2}} \right)}} \mathop \sum \limits_{j = 1}^{n} \left( {\left| {i - \frac{m}{2}} \right| \cdot \left| {j - \frac{n}{2}} \right|} \right) \cdot \frac{{\left| {p\left( {i,j} \right) - p\left( {i + 1,j} \right)} \right|}}{{\sqrt {\left( {p\left( {i,j} \right)} \right)^{2} + \left( {p\left( {i + 1,j} \right)} \right)^{2} } }}} \right) \hfill \\ + \left( {\mathop \sum \limits_{i = 1}^{m} \mathop \sum \limits_{j = 1}^{{round\left( {\frac{n}{2}} \right)}} \left( {\left| {i - \frac{m}{2}} \right| \cdot \left| {j - \frac{n}{2}} \right|} \right) \cdot \frac{{\left| {p\left( {i,j} \right) - p\left( {i,j + 1} \right)} \right|}}{{\sqrt {\left( {p\left( {i,j} \right)} \right)^{2} + \left( {p\left( {i,j + 1} \right)} \right)^{2} } }}} \right) \hfill \\ \end{aligned} \right) $$
(20)

Above feature F 18 is a modified version of the so called “Cosine similarity” indicator used to analyze the intrinsic similarity of the nevus. The features from F 19 to F 21 have been computed as results of further SC-CNNs post-processing performed in the “SC-CNNs Hand Crafted Block”. The SC-CNNs programmed via “edge detection” cloning templates [14, 15] is used to detect edges of the gray-level segmented image \( Y_{SC - CNN(x,y)}^{\prime } \). Let denotes the so processed image as \( Y_{SC - CNN(x,y)}^{pp} \). Considering same parameters as reported in Fig. 5 but related to image \( Y_{SC - CNN(x,y)}^{pp} \), we compute the latest set of hand-crafted image features:

$$ F_{19} = { \log }\left( {\frac{1}{m \cdot n}\mathop \sum \limits_{i = 1}^{mr} \mathop \sum \limits_{j = 1}^{nr} p\left( {i,j} \right)} \right) $$
(21)
$$ F_{20} = { \log }\left( {\pi \cdot \left( {\frac{nr - 1}{2}} \right)^{2} } \right) $$
(22)
$$ F_{21} = \frac{{{ \hbox{min} }\left( {mr,nr} \right)}}{{{ \hbox{max} }\left( {mr,nr} \right)}} $$
(23)

2.3 The LV Network Clustering System

After we have collected the numerical values corresponding to morphological-heuristic e hand-crafted features, we are ready to perform a skin lesion classification needed for oncological risk assessment of the analyzed nevus i.e. we provide a classification of the analyzed skin lesion as “benign nevus” or “melanoma” (skin cancer). According to skin lesion membership of one of the above two classes, the algorithm provides follow-up rate according to ad-hoc heuristic risk assessment specifying a time-rate of periodic medical check-up of the analyzed lesion in order to monitoring time-evolution of the lesion by means of the numerical array of features F 1 …F 21 as per above description.

The normalized set of proposed innovative image features F 1 …F 21 are used as input of ad-hoc Feed-Forward Levenberg-Marquardt (LV) Optimized Neural Network previously pre-trained with carefully selected skin lesions training set containing lesions with different morphological features (color, border, geometry, irregularities, etc.) both benign and malignant(melanoma). We designed and trained that LV Feed-Forward Neural Network with one hidden layer: the neural network dimension is 21 × 25 × 2. The output of LV Neural Network with value close to [1, 0] will be considered as benign nevus while [0, 1] will be considered as melanoma. The used neural network is trained with efficient optimized Levenberg-Marquardt back-propagation learning algorithm [15, 16] with stop criteria correlated to learning mean square error i.e. it Several works in literature have proposed the usage of Artificial Neural Network or Convolutional Neural Networks (CNNs) in order to address the target of early detection of skin cancer by analysis of dermoscopy images [5,6,7]. In our experiments in order to improve the generalization capability of the neural network as well as for avoiding the issue of local convergence of training algorithm, we have defined the training set carefully introducing selected dermoscopy images with different patterns (colors, borders, irregularities, benign lesions, melanoma, etc.). After that we have splitted training set in a part for training while remaining part has been used for testing/validation of the network. In our setup we have used MATLAB with Image and Neural Network toolboxes. The used dermoscopy images for training set have been taken partially from PH2 database [10] and partially from other medical database provided us by oncologists. All the source images will be resized to 768 × 576 pixels by using same algorithm proposed in [8].

2.4 The Autonomous Diagnosis System

The numerical output of the LV Neural Network together with hand-crafted features is also used for determining the follow-up rate of the analyzed nevus, by using heuristic thresholds based rules as followed described. We define a set of thresholds for the proposed mainly discriminative hand-crafted image features i.e. the features F 12 , F 17 , F 21 . We define those thresholds by means of averaging computation of the values heuristically computed during the training phase of the LV Neural Network according to oncologist advices:

$$ Th_{{F_{12} }} = 12,99 ; Th_{{F_{17} }} = 104,00; Th_{{F_{21} }} = 6,49 $$

The above thresholds are used to define ad-hoc follow-up rate according to nevus diagnosis performed by LV Neural Clustering System. Specifically, the Autonomous diagnosis system for “melanoma diagnosis” suggests “Contact Physician as soon as possibile” in case of both features F 12 , F 17 , F 21 are greater than related thresholds while extend the follow-up to “1 month” otherwise (at least one of the above features is less of the corresponding thresholds) because even the diagnosis is “melanoma” with high probability, the exanimated nevus does not show high malignant medical indexes according to oncologists consultation. As per “melanoma diagnosis”, in case of “benign nevus” diagnosis performed by LV Neural Clustering System, the Autonomous Diagnosis System suggests “Follow-up rate ≥ 1 year” for both main features (F 12 , F 17 , F 21 ) within the corresponding thresholds while it reduces the time-range “Follow-up rate ≥ 6 months” in case of at least one of the main features is greater than corresponding thresholds.

3 Experimental Results and Future Works

The proposed method has been validated by using the skin lesions database named “PH2” kindly provided within the ADDI project [3, 10]. The PH2 database consists of 176 dermoscopy images carefully classified in the related project webpage [3, 10]. The performance of the proposed pipeline has been compared with algorithms proposed in the ADDI project [3, 10] and validated with PH2 database. In order to get easily the comparison between the proposed approach and other ones, we have computed same benchmark indicators i.e. Sensibility (SE) and Specificity (SP) and cost function “C” computed as reported in [3, 10]:

$$ \begin{aligned} C = \frac{{c_{10} (1 - SE) + c_{01} (1 - SP)}}{{c_{10} + c_{01} }} \hfill \\ c_{10} = 1.5c_{01} ;\,c_{01} = 1 \hfill \\ \end{aligned} $$
(24)

In (24) we defined c 10 as the weight coefficient of incorrect classification of a melanoma as benign nevus (a false negative FN) while we indicating c 01 as the weight coefficient for incorrect classification of benign nevus as melanoma (a false positive FP). From values reported in (24), it is clear we consider a missed classification of melanoma more dangerous with respect to a wrong classification of benign nevus as melanoma. Table 1 reports the performance benchmark comparison [3, 10].

Table 1. Benchmarks comparison for the proposed pipeline
Full size table

The simple comparison between benchmarks reported in Table 1, shows the very promising performance of the proposed method with respect to other ones proposed in literature. An innovative combination between ad-hoc innovative hand-crafted image features with supervised neural networks properly trained has improved drastically the ability of the overall discrimination system in identifying the key nevus features for a robust classification of benign lesions against high suspected ones or melanoma cancer (see Table 2 for results). Often for improving the ability of their pipelines, some authors increase sensibility reducing the overall checks of the skin lesion image statistics. The drawback of poor specificity of the pipeline means more nevus biopsy for the patients.

Table 2. Some instances of classified nevus
Full size table

The time performance of the proposed method is acceptable as the proposed pipeline is able to analyze a single nevus in about 2.5 s (we are testing the pipeline in a PC Intel Core i5 3.10 GHz @ 64 bit with 4 Gbyte of RAM). The proposed pipeline can be ported easily from MATLAB environment to an embedded platform based on STM32 [17]. We proposed a very efficient method for skin nevus analysis and oncological classification both for screening and follow-up of exanimated skin lesion. We are extending skin lesions image dataset in order to cover all the specific skin lesion features with aim to perform a more precise clustering including further class of “suspected” nevus often one of the main issue for dermatologists and oncologists.