Colored Texture Analysis Fuzzy Entropy Methods with a Dermoscopic Application

Texture analysis is a subject of intensive focus in research due to its significant role in the field of image processing. However, few studies focus on colored texture analysis and even fewer use information theory concepts. Entropy measures have been proven competent for gray scale images. However, to the best of our knowledge, there are no well-established entropy methods that deal with colored images yet. Therefore, we propose the recent colored bidimensional fuzzy entropy measure, FuzEnC2D, and introduce its new multi-channel approaches, FuzEnV2D and FuzEnM2D, for the analysis of colored images. We investigate their sensitivity to parameters and ability to identify images with different irregularity degrees, and therefore different textures. Moreover, we study their behavior with colored Brodatz images in different color spaces. After verifying the results with test images, we employ the three methods for analyzing dermoscopic images of malignant melanoma and benign melanocytic nevi. FuzEnC2D, FuzEnV2D, and FuzEnM2D illustrate a good differentiation ability between the two—similar in appearance—pigmented skin lesions. The results outperform those of a well-known texture analysis measure. Our work provides the first entropy measure studying colored images using both single and multi-channel approaches.


Introduction
Texture features are of the utmost importance in segmentation, classification, and synthesis of images, to cite only few image processing steps. However, no precise definition of texture has been adopted yet. Texture is often referred to as the visual patterns appearing in the image. Several algorithms have been proposed for texture feature extraction in recent years and this research area is still the subject of many investigations [1][2][3][4][5][6][7][8][9][10]. Recently, seven classes were proposed to classify the texture feature extraction methods [1]: statistical approaches (among which we can find the co-occurrence matrices), structural approaches, transform-based approaches (Fourier transform-based approaches, among others), modelbased approaches (such as the random field models), graph-based approaches (such as the local graph structures), learning-based approaches, and entropy-based approaches. The latter two classes (learning-based approaches and entropy-based approaches) are the most recent ones. Several studies have shown that the entropy-based measures are promising for texture analysis [11][12][13][14][15][16][17][18]. However, these studies are only at their beginning. Even if they have the great advantage of relying on reliable unidimensional, 1D, entropy-based measures (issued from the information theory field), they have the drawback-for most of them-of being designed for gray scale images only.
Besides texture, color is essential not only for human perception of images but also for digital image processing [19][20][21][22][23][24][25]. Unlike the intensity that is translated as scalar gray values for a gray scale image, color is a vectorial feature that is appointed to each pixel for a colored

Colored Bidimensional Fuzzy Entropy
We recently developed bidimensional fuzzy entropy, FuzEn 2D , and its multi-scale extension MSF 2D [17,18,38]. These entropy measures revealed interesting results for some dermoscopic images but were limited to gray scale images. Based on FuzEn 2D , we propose herein approaches to deal with colored images: the single-channel bidimensional fuzzy entropy, FuzEnC 2D [28] which considers the characteristics of each channel independently, and the multi-channel bidimensional fuzzy entropy measures, FuzEnV 2D and FuzEnM 2D , which take into consideration the inter-channel characteristics. In this paper, we limit our study to three color channels. However, extension to a higher number of channels would be straightforward. For a colored image U of W width, H height, and K channels (W × H × K pixels), the following initial parameters are first set: tolerance level r, fuzzy power n, and window size m (see below). The algorithms to compute FuzEnC 2D , FuzEnV 2D , and FuzEnM 2D are presented below.

FuzEnC 2D Single-Channel Approach
The colored image U is separated into its corresponding color channels K1, K2, and K3, as U K1 , U K2 , and U K3 , respectively. For each channel composed of u K (i, j) elements, X m i,j,K is designated as the m-length square window: with K = K1, K2, or K3 and the indices are defined as such: i,j,K , is defined in the same way. In each of U K1 , U K2 , and U K3 , the total number of defined square windows for both m and m + 1 sizes is Based on the original fuzzy entropy definition, FuzEn 1D [39], a distance function d m ij,ab,K between X m i,j,K and its neighboring windows X m a,b,K is defined as the maximum absolute difference in their corresponding scalar components. We compose d m ij,ab,K as follows: with a ranging from 1 to H − m and b ranging from 1 to W − m. The similarity degree D m ij,ab,K of X m i,j,K with its neighboring patterns X m a,b,K is defined by a continuous fuzzy function µ(d m ij,ab,K , n, r): D m ij,ab,K (n, r) = µ(d m ij,ab,K , n, r) = exp(−(d m ij,ab,K ) n /r).
Afterwards, the similarity degree of each X m i,j,K is averaged to obtain Φ m i,j,K (n, r) and then construct: It is similar for m + 1 patterns to obtain Φ m+1 K (n, r). Consequently, FuzEn 2D of each channel is calculated as: Finally, FuzEnC 2D is defined in each channel as the natural logarithm of the conditional probability that patterns with m × m similar pixels would remain similar for the next (m + 1) × (m + 1) pixels in each channel: This single-channel approach treats each channel independently. It has the advantage of allowing us to selectively study certain channels which is of special importance when it comes to images in different color spaces and natures (intensity, color, and texture). In our study, we used n = 2. Thus, the similarity degree is expressed by a Gaussian function exp(−(d m ij,ab,K ) 2 /r). For better illustration, we show in Figure 1 an example for FuzEnC 2D of an RGB color space image for an embedding dimension of m = [2,2]; i.e., m × m pixels for each channel. The illustration shows RGB channels as an example, but the same could be applied to different color spaces.

FuzEnV 2D Multi-Channel Approach
For an image U composed of u i,j,k pixels, X m i,j,k is defined as the m-length cube. X m i,j,k represents the group of pixels in the image U of indices from line i to i + m − 1, column j to j + m − 1, and the depth of K-channels (k: depth index) as follows: Similarly, X m+1 i,j,k is defined as the (m + 1)-length cube. Let N m = (W − m)(H − m) (K − m) be the total number of cubes that can be generated from U for both m and m + 1 sizes. For X m i,j,k and its neighboring cubes X m a,b,c , the distance function d m ijk,abc between them is defined as the maximum absolute difference of their corresponding scalar components, knowing that a, b, and c range from 1 to H − m, W − m, and K − m, respectively. Having (a, b, c) = (i, j, k), the distance function is depicted as follows: The similarity degree D m ijk,abc of X m i,j,k with its neighboring cubes X m a,b,c is defined by a fuzzy function µ(d m ijk,abc , n, r): D m ijk,abc (n, r) = µ(d m ijk,abc , n, r) = exp(−(d m ijk,abc ) n /r).
Afterwards, the similarity degree of each cube is averaged to obtain Φ m i,j,k (n, r), then construct: This is similar for m + 1 cubes to obtain Φ m+1 (n, r). Finally, multi-channel bidimensional fuzzy entropy of the colored image U is defined as the natural logarithm of the conditional probability that cubes similar in their m × m × m pixels would remain similar for the next (m + 1) × (m + 1) × (m + 1) pixels: The multi-channel approach has the advantage of extracting inter-channel features. However, we limit our study herein to 3-channel colored images. Thus, the embedding dimension m values could be 1 or 2 to avoid exceeding the maximum possible 3 × 3 × 3 pixels cubes for the m + 1 calculations. This means that for K channels the m-value can only be defined between 1 and K-1. Herein, n is taken to be 2 and r within the range suggested in previous studies. For better illustration, we show in Figure ] that is a 2 × 2 × 2 cube; (c) X m i,j,k and X m a,b,c , the fixed and moving templates defined above.

FuzEnM 2D Modified Multi-Channel Approach
Since the FuzEnV 2D embedding dimension size is limited to m = 1 and m = 2 for this trichromatic study (K = 3), we introduce herein a modified colored multi-channel approach that can take up to any m value. This method is similar to FuzEnV 2D except for the fact that the embedding dimension is a cuboid of m × m × K voxels for FuzEnM 2D . Therefore, the third dimension of the template is not limited by the number of color channels in the study.
For image U with K = 3 color channels, composed of u i,j,k voxels, X m i,j,k is defined as the m × m × 3 cuboid. X m i,j,k represents the group of voxels in the image U of indices from line i to i + m − 1, column j to j + m − 1, and the depth of K-channels (k: depth index).
be the total number of cuboids that can be generated from U for both m and m + 1 sizes. Sizes m and m + 1 stand for [m, m, 3] and [m + 1, m + 1, 3] that are made up of m × m × 3 and (m + 1) × (m + 1) × 3 voxels, respectively.
For X m i,j,k and its neighboring cuboids X m a,b,c , the distance function d m ijk,abc between them is defined as the maximum absolute difference of their corresponding scalar components, knowing that a and b range from 1 to H − m and W − m, respectively, whereas c is 1. Having (a, b, c) = (i, j, k), the distance function is depicted as follows: The similarity degree D m ijk,abc of X m i,j,k with its neighboring cuboids X m a,b,c is defined by a fuzzy function µ(d m ijk,abc , n, r): Afterwards, the similarity degree of each cuboid is averaged to obtain Φ m i,j,k (n, r), then construct: This is similar for (m + 1) × (m + 1) × 3 cuboids to obtain Φ m+1 (n, r). Finally, multi-channel bidimensional fuzzy entropy of the colored image U is defined as the natural logarithm of the conditional probability that cuboids similar in their m × m × 3 voxels would remain similar in their (m + 1) × (m + 1) × 3 voxels: FuzEnM 2D has the advantage of extracting inter-channel features and always considering all the color channels of texture images. However, as mentioned previously, we consider our study herein for 3-channel colored images which could be further adapted to a higher number as well. Herein, n is taken to be 2 and r within the range suggested in previous studies. For better illustration, we show in Figure 3 an example for FuzEnM 2D of an RGB color space image for an embedding dimension of m = [2, 2, 3]; i.e., moving m-sized cuboid is 2 × 2 × 3.

Comparing Algorithms
The proposed entropy measures are based on the fuzzy entropy definition [17,39,40] that calculates the similarity degree between the corresponding patterns using a continuous fuzzy function. The latter ensures calculating a participation degree for all the compared patterns and quantifies the irregularity of the analyzed data. This information theory concept has been proven to be reliable for 1D, 2D, and 3D data [17,18,[38][39][40]. However, only gray scale data have been investigated to date. Therefore, the idea to analyze colored texture images using the fuzzy entropy concept from a single channel and a multi-channel perspective is interesting. The major differences between the three proposed algorithms are in the way the similarity degrees are calculated. For the single-channel approach, FuzEnC 2D , the image is analyzed channel by channel and the result is three entropy values that represent the three channels, respectively, please refer to Figure 1. This is a particular advantage when it comes to analyzing and comparing specific channels in different color spaces. On the other hand, the multi-channel approaches, FuzEnV 2D and FuzEnM 2D , deal with all the channels at the same time; i.e., the inter-channel information is taken into account (unlike handling each color channel separately). FuzEnV 2D transforms the 2D similarity degree scanning window into a 3D cubic pattern that studies similarity among the m × m × m and the m + 1 × m + 1 × m + 1 patterns within a colored image. FuzEnV 2D showed good results but for the application in trichromatic color spaces the embedding dimension size was limited to m = 1 or 2, please see Figure 2. Therefore, in order to investigate similarity degrees with larger embedding dimension sizes, we present the modified multi-channel approach FuzEnM 2D , please refer back to Figure 3. FuzEnC 2D , FuzEnV 2D , and FuzEnM 2D provide colored texture analysis from single-channel and multi-channel perspectives. The choice of the algorithm depends on the intended application. Moreover, the analysis could be extended to multi-spectral images and even other color spaces than the ones discussed in this paper.

Validation Tests and Medical Database
In order to validate the proposed colored bidimensional entropy measures, we studied their sensitivity to different parameter values. The algorithms were also tested using images with different degrees of randomness and the colored Brodatz dataset [41]. The images were normalized by subtracting their mean and dividing by their standard deviation and all the tests were performed using MATLAB. In the following, we describe the elements used for the validation tests and the medical dataset.

MIX 2D (p) Processes
MIX 2D (p) [12] is a family of images of stochastic processes that are moderated by the probability of irregularity, p, varying from 0 (totally regular periodic image) to 1 (totally irregular image). We used MIX 2D (p) for the single-channel approach, and MIX 3D (p), a volumetric extension for MIX 2D (p) proposed by [40], for our multi-channel approach.

Colored Brodatz Images
For texture validation tests, we used the colored Brodatz texture (CBT) [41,42] images, see Figure 4. CBT presents colored textures with different degrees of visible irregularity.
We can notice that, for example, the CBT images (a), (b) and (e) show more regular and periodic repetitive patterns than (c), (f) and (i).

Color Spaces
Besides using the most common trichromatic color space, red, green, blue (RGB), we extend our study by transforming the images to use two other color spaces: hue, saturation, value (HSV; hue and saturation: chrominance, value: intensity) and YUV (Y: luminance, U and V: chrominance) to investigate the effect of color space transformations on FuzEnC 2D , FuzEnM 2D , and FuzEnM 2D outcomes. In RGB color space, the intensity and color are combined to give us the final display, whereas for HSV and YUV color spaces, intensity and color are separated.

Co-Occurrence Matrices
For the application on medical images, we study the effect of different color spaces and compare our results to those obtained with gray level co-occurrence matrices [43], which probably remains the most used texture analysis technique. We employed the co-occurrence matrices of each channel (integrative way) for comparing the results to our single-channel approach, and its extended 3D co-occurrence matrices [44] for comparing the results to our multi-channel approach. We thus adopted the following procedure: • The 2D co-occurrence matrices were created considering 4 orientations (0 • , 45 • , 90 • , and 135 • ), 4 inter-pixel distances (1, 2, 4, and 8), and 8 gray levels (N g = 8) to be compared with FuzEnC 2D . • The 3D co-occurrence matrices were created considering 13 orientations [44], 4 interpixel distances (1, 2, 4, and 8), and 8 gray levels to be compared with FuzEnV 2D and FuzEnM 2D .
Then, we calculated the Haralick features for each co-occurrence matrix (for each orientation and distance). Finally, the average of features for all matrices was calculated to be compared with FuzEnC 2D , FuzEnV 2D , and FuzEnM 2D values. Among the 14 features originally proposed [43], only six are commonly employed by researchers due to their correlation with the other eight, see Table 1.

Haralick Feature Annotation
where P represents the elements of the co-occurrence matrices and µ x , µ y , σ x , and σ y are the means and standard deviations of row and column sums, respectively.

Medical Images
For our medical application we used the HAM10000, "Human Against Machine with 10,000 training images" [45,46]. The dataset is composed of dermoscopic images for pigmented lesions, see an example in Figure 5a. The dataset contains dermoscopic images of melanocytic nevi, melanoma, dermatofibroma, actinic keratoses, basal cell carcinoma, and benign keratosis [45].
As suggested by medical doctors, the most significant comparison is that between melanoma and melanocytic nevi. The target of the medical application in our study is to try to differentiate the deadliest type of skin cancer, melanoma, from the benign melanocytic nevi. These two widespread types of pigmented skin lesions are often mistaken in diagnosis and detection, especially in their early stages. Moreover, early diagnosis and excision could vastly increase the patients' survival rate [29][30][31]. Thus, we selected from the proposed dataset forty melanoma images and forty melanocytic nevi images to be processed and compared.

Results and Discussion
In this section, we present the results of the validation tests. We start by testing the algorithms' sensitivity to initial parameter choice, then we explore the algorithms' ability to identify increasing irregularity degrees in colored textures. After that, we analyze colored Brodatz texture images in 3 different color spaces (RGB, YUV, and HSV). Finally, we show the results using FuzEnC 2D , FuzEnV 2D , and FuzEnM 2D for melanoma and melanocytic nevi dermoscopic images and compare them to those obtained using single-channel and multi-channel co-occurrence matrices.

Sensitivity to Initial Parameters
To study the sensitivity of our proposed measures, with different embedding dimensions m and tolerance levels r, we evaluated 100 × 100 pixels of a colored Brodatz image (Figure 4f) using different parameter choices.

•
For FuzEnC 2D , the embedding dimension m was taken as 1, 2, 3, 4, and 5, and the tolerance level r from 0.06 up to 0.48 (step 0.06). The results are displayed in Figure 6. • For FuzEnV 2D , the embedding dimension m was taken as 1 and 2, since the maximum possible cube volume for (m + 1)-length cubes is 3 × 3 × 3 pixels (given the 3 color channels). The results are displayed in Figure 7.  We observe that FuzEnC 2D , FuzEnV 2D , and FuzEnM 2D remain defined for different chosen initial parameters. Additionally, the algorithms show low variability upon changes in r and m. This illustrates their low sensitivity to r and m, allowing a certain degree of freedom in our choice of initial parameters without restrictions.

•
FuzEnC 2D : we set r = 0.15, m = 1, 2, 3, 4, 5, and p = 0 to 1 with a step of 0.1, and repeated the calculation for 10 images each. The results are depicted in Figure 9. • FuzEnV 2D : we set r = 0.15, m = 1 and 2 (as the maximum possible cube volume for m + 1 could only be 3 × 3 × 3 pixels), p = 0 to 1 with a step of 0.1, and repeated the calculation for 10 images each. The results are depicted in Figure 10. • FuzEnM 2D : we set r = 0.15, m = 1, 2, 3, and 4, and p = 0 to 1 with a step of 0.1, and repeated the calculation for 10 images each. The results are depicted in Figure 11. The results show that both the single-and multi-channel approaches lead to increasing entropy values with increasing irregularity degree, p. This illustrates their ability to properly quantify increasing irregularity degrees and their consistency upon repetition.

Studying Texture Images
Nine CBT [41,42] images of 640 × 640 pixels, see Figure 4, were split into 144 subimages of size 50 × 50 pixels. FuzEnC 2D , FuzEnV 2D , and FuzEnM 2D were calculated for these sub-images and for a 300 × 300 pixel corner region from each corresponding original CBT image. The parameters r and m were set to 0.15 and 2, respectively. The results with FuzEnC 2D and FuzEnV 2D are depicted in Figures 12 and 13. Similar results to those of FuzEnV 2D are found with FuzEnM 2D . We observe that, especially for the RGB color space, most of the FuzEnC 2D , FuzEnV 2D , and FuzEnM 2D averages of the sub-images overlap with or are very similar to the value of their corresponding image's 300 × 300 pixel region. Moreover, we notice their differentiation ability between different CBT images. In the HSV and YUV color spaces, the multi-channel approaches outperform FuzEnC 2D (Figure 12) in differentiating the CBT images. We can also observe that for the RGB color space, the CBT images that are perceived visually to be of higher color and pattern irregularity, Figure 4c,f,g, obtained higher entropy values than the others, whereas those that appear to be of periodic well-defined repetitive patterns, Figure 4a,b,e, resulted in lower entropy values for the three measures FuzEnC 2D , FuzEnV 2D , and FuzEnM 2D . This is in accordance with the literature of entropy measures and information theory concept applied to gray level texture images [12,[14][15][16][17][18]38]. Figure 12. FuzEnC 2D results for the 144 sub-images and 300 × 300 pixels of the CBT in the three color spaces: RGB, HSV, and YUV, with K1, K2, and K3 being the first, second, and third channel, respectively. The mean of the 144 sub-images is displayed as a "•" sign and the value for the 300 × 300 pixels is displayed as " * ". Figure 13. FuzEnV 2D results for the 144 sub-images and 300 × 300 pixels of the CBT in the three color spaces: RGB, HSV, and YUV. The mean of the 144 sub-images is displayed as a "•" sign and the value for the 300 × 300 pixels is displayed as " * ".

Medical Image Analysis
We calculated FuzEnC 2D , FuzEnV 2D , and FuzEnM 2D for 40 melanoma images and 40 melanocytic nevi images from the HAM10000 dataset [45] in the color spaces RGB, HSV, and YUV. In order to determine the region of interest (ROI) of melanoma and melanocytic nevi images, the lesions were segmented as shown in Figure 5. Then, the central region of 128 × 128 × 3 pixels was selected, see Figure 5d. By adopting this procedure, we ensured that the same number of pixels were processed (equally sized images) and that no region outside the lesion was included. The parameters r and m were set to 0.15 and 2, respectively. The images were normalized by subtracting their mean and dividing by their standard deviation.
To validate the statistical significance of FuzEnC 2D , FuzEnV 2D , and FuzEnM 2D in differentiating melanoma from melanocytic nevi images, we used the Mann-Whitney U test. The resulting p-values are presented in Table 2. FuzEnC 2D shows statistical significance (for p < 0.05) in differentiating melanoma and melanocytic nevi for all the channels except V (of HSV color space). In addition, using FuzEnV 2D and FuzEnV 2D , melanoma and melanocytic nevi images are identified as statistically different for the three color spaces. Moreover, we calculated the Cohen's d [47,48] to further validate our obtained statistical results, see Table 3. Most d values reflect "large", "very Large", and "huge" effect sizes, which validates the differentiation ability of our proposed measures. Table 2. Mann-Whitney U test p-values for FuzEnC 2D , FuzEnV 2D , and FuzEnM 2D of 40 melanoma and 40 melanocytic nevi dermoscopic images in the 3 color spaces: RGB, HSV, and YUV, from top to bottom row, respectively.   In addition to the p-value test, the receiver operating characteristic (ROC) and area under the ROC curve (AUC) of the results can be used as a criterion to measure the discrimination ability of our proposed measures. Since the best results (lowest p-values) were obtained for the RGB color space, we further establish the ROC curves for its FuzEnC 2D , FuzEnV 2D , and FuzEnM 2D results, see Figures 16, 17 and 18, respectively. Moreover, the AUC, sensitivity, specificity, accuracy, and precision are shown for the RGB, HSV, and YUV color spaces in Tables 4, 5 and 6, respectively. The results show that FuzEnC 2D has high accuracy and AUC values for R, G, B, H, Y, U, and V channels. In addition, the multi-channel approaches (FuzEnV 2D and FuzEnM 2D ) illustrate high accuracy and AUC values for the three color spaces. For the three proposed entropy measures, the best accuracy and AUC values were obtained for the RGB color space.

Conclusions
In this paper, we presented a new concept and the first entropy method to investigate the single-and multi-channel features of colored images. To the best of our knowledge, this study is the only one that suggests entropy measures for analyzing colored images in their single-and multi-channel approaches. It was essential to perform some validation tests before employing those measures for analyzing colored medical images. The study was carried out as follows: • Studying the sensitivity of the proposed measures to different initial parameters (tolerance level r and window size m).

•
Identifying different irregularity degrees in colored images. • Studying colored texture images in three color spaces. • Analyzing medical images in three color spaces.
The three entropy measures, FuzEnC 2D , FuzEnV 2D , and FuzEnM 2D , showed a reliable behavior with different initial parameters and an ability to gradually quantify irregularity degrees of colored textures and consistency upon repetition. When considering different color spaces, RGB, HSV, and YUV, these entropy measures showed promising results for the colored texture images.
Regarding the dermoscopic melanoma and melanocytic nevi images, single-and multi-channel entropy measures were able to differentiate both pigmented skin lesions. This was validated statistically by p-values, especially in the RGB color space. In the latter, FuzEnC 2D achieved accuracies of 83.7%, 88.7%, 86.2% and AUC of 88.4%, 94.5%, 93%. On the other hand, FuzEnV 2D , reached an accuracy of 93.7% and AUC of 96.4%. In addition, FuzEnM 2D showed an accuracy of 91.2% and AUC of 95.0%. Moreover, FuzEnV 2D and FuzEnM 2D outperformed both FuzEnC 2D and the classical descriptors, Haralick features, in differentiating the two similar malignant melanoma and benign melanocytic nevi dermoscopic images. These preliminary results could be the groundwork for developing an objective computer-based tool for helping medical doctors in diagnosing melanoma that is often mistaken for a benign melanocytic nevi or is properly diagnosed only in its late stages. We limited our investigation to three-channel colored images and, consequently, future work could be directed towards multi-spectral color images and towards more adapted applications for each color space and extending our study to a larger dataset.