Acne detection, assessment, grading and classification using machine learning techniques: a review

Acne is one of the most common problems faced by huge population (above 90%) at different age groups, genders and different area of acne and its severity. Among all acne types, acne vulgaris is most common of them. Acne vulgaris has become an interesting domain for researchers in biomedical engineering as well as image processing. Recognizing acne region and skin areas accurately is really challenging task.This plays a major role in acne detection, grading, classification, acne severity detection and automatic acne assessment. This paper presents a comprehensive review which aim to fill the research gap in literature by providing all the state-of-the-art methods applied till date on acne vulgaris images. This research area is least explored and hence this paper focuses on survey of various image processing and machine learning techniques applied on acne images. Future scope and the problems identified in this domain are also elaborated.


Introduction
Irrespective of age and gender acne is a skin disease that afflict many people but more generic in teenagers (85% of overall population) [1]. A review on different acne types and assessment methods are elaborated in [2] which includes following five main types in Acne Vulgaris ascyst, papule, pustule, nodule and comedown. Acneis more severely found on face, chest and upper back. This paper also includes a brief description on the types of acne vulgaris. Since beauty industry is one of fastest growingindustry at present time along with acne being a very common disease with plenty of acne patients, the most benefited service for beauty clinic is acne treatment [3]. Since, conventional methods for the assessment of acne vulgaris by dermatologists are lesion counting and other is comparison ofpatients' acne image and its reference image [4]. In both assessment methods manual interference is involved that triggers subjectivity. Hence different acne detection, skin detection, acne assessment grading and classifications methods are proposed in the literature. Other problem observed is the phenomenon of specular illumination in the imaging of acne patients because of the Oily skin nature, pus inside Acne lesions and medication applied to faces and many more. Different pre-processing methods are also proposed in research paper that gives methods to remove the specular illumination and other lighting condition effects. This review paper covers the gap in the literature as the last review paper in the given domain by found in the year 2012 by Ramli et.al. [5]. The authors elaborated types of acne, its causes, conventional acne assessment methods, computational acne assessment methods and conclusion and future scope. The grading system of the acne is one of the main issues as no global standardized grading system can be found because of variety in color complexions. Moreover, it is a subjective measure as it depends on dermatologist to dermatologist along with variation by the same dermatologist at different time [5]. In this paper, the detailed analysis of acne grading system as well as measurement has led to the identification of issues related to acne that need to be addressed for effective acne treatment. But with the state-of-the-art techniques with the new hikes in machine learning [6], artificial intelligence and computer vision various new and improved systems are designed. Deep learning is now present in a wide range of services and applications, replacing and complementing other machine learning algorithms. Over the last thirty years, Deep Learning (DL) algorithms have evolved very fast and have become promising algorithms with better results than other previous machine learning approaches [7]. Nonetheless, DL depends on the availability of high-performance computing platforms with a large amount of storage, required for the data needed to train these models [8]. Deep neural network finds a lot of applications in image and video analysis [9]. Using Convolution neural network (CNN)[10] one of the Deep learning architectures can provide even better and accurate classification and grading system of such images. There are remarkable number of search engines and academic databases available and many of the research articles are taken from Web of Sciencee, Institute of Electrical and Electronics Engineers (IEEE) Xplore, Science Direct, and Association for Computing Machinery (ACM) Digital Library for writing this review article. Papers that are highly cited are preferred than least cited articles. The review paper is structured as follows: section 2 provides a brief description on the types of acne vulgaris. Section 3 provides a survey on the work done in this domain.A brief discussion on issues associated with the given approaches is also included. Section 4 elaborates the datasets available in hand for this domain some others methods adopted by the users. Future work and conclusion are made in the last section.

Acne Types
In general, there are different types of acne possible namingly: acne conglobata, acne excoriee, acne cosmetica (due to cosmetics usage), pomade acne (due to talcum powder), acne fulminans, acne mechanica, acne medicamentosa (due to tropical medicine), etc. [11]. Acne vulgaris is the most common accounting 99% of the acne cases and hence there assessment and classification is more preferred. These types of acne can be further classified into different types such as: comedones (whitehead or blackheads), papules, pustules, nodules, cysts or scarring. Acne vulgaris is characterized by non-inflammatory as well as inflammatory types. Open and closed comedones are non inflammatory in nature whereas papules, pustules and nodules are inflammatory.

Figure-1 Acne lesions. (a) Whitehead (closed comedones), (b) milia, (c) blackhead (open comedones), (d) papules, (e) pustules (f) nodules (g) cysts. [11]
Comedones are further classified as a whitehead (a closed comedo) and a blackhead (an open comado). A whitehead is an acne lesion that appears on the skin as a whitish bump that is under skin surface as shown in Fig. 1 (a). A blackhead is an acne lesion that is appears as a black open comedo as shown in Fig. 1 (b). Papules can be seen as a small pinkish or red bump ( Fig. 1(c)) whereas pustules are full of visible pus between the red base with a yellow/white center that can very much inflammable ( Fig. 1 (d)). A nodule (5-10 mm in size) is similar to a papule but more in its size is shown in Fig. 1(e). Cysts are the largest of the types of acne vulgaris lesions. These are very painful, re inflammatory and can also leave deep scars ( Fig. 1(f)).
The most common among all the acne vulgaris types is Papules and pustules. It is not necessary that a patient will have only type of acne but instead they may have a combination two or more types of it. Table 1 provides the descriptive features of each types of acne lesions discussed in this section.

Literature Review
Acne images need different preprocessing and one such work to remove the specular illumination from the acne images was done in [4]. The texture and color of objects depends on lighting conditions as well as on the orientation of scene and light source [12].In this paper they detected the specularly illuminated regions and recovered their color information based on close neighbors. Specular surfaces reflect maximum light and hence appear totally white regions and this makes them to use the energy calculation, a suitable measure for the detection of specularity in images. After detection, recovery of the color information is done based on the feature that a group of pixels contribute to objects have similar attributes and hence specularly illuminated pixel were assigned the most probable color from its nearest neighbors. For this purpose, a window of suitable size was drawn around the pixel affected with specular illumination and inside the window using the histogram of pixels (except the specularly effected pixels) the most probable color was found and replaced for those specularly effected pixels. Figure 2 shows the work done by them.

Figure-2 Images (a), (b), (c) are the original images while (d), (e), (f) are their respective processed images [4].
In 2016, NatchapolKittigul et.al [13], proposed a system which first extracted RGB channels from original image and converted them to gray scale image. Using the Haar Cascade classifier they detected and extracted facial region. This facial region is displayed by a red rectangular box overlayedthis region. They removed the background using the GrabCut marking foreground as ROI. Face detection was completed by this step. Adaptive thresholding was done on the Green channel of cropped facial-region image and BLOB detection was done which detected some white BLOB on black background. This detection was used to represent acne on the face and it provided promising results. At every BLOB a red rectangle was drawn over them and this method was tested on patient's cheek image. The authors wanted to validate the results by extracting features from each BLOBs and compared with some set of acne features in order to classify the BLOB as acne or non-acne for future work. In this paper [14], the authors proposed a computer aided diagnosis system to extract acne lesions of various types using multispectral imaging. As pre-processing authors removed the effect of shade and gloss and then used spectral information of each pixel to do the classification. In this paper the authors, classify the four types of acne lesions as come do, reddish papule, pustule and scar. According to the authors the color image (RGB) is not enough for acne lesions detection and hence multispectral imaging is used. Classification is done using combination of linear discriminant functions (LDF's). Using given method an initialevaluation of the severity of Acne can be achieved. They had some issues in their process which they will address in future.
In [15], paper aimed to propose a fully automated acne detection and diagnosis application by processing of distance picture at home of any body parts of the patient imagesfrom the mobile phones. They tested their system using sseveral classifiersand compared among Logistic Regression, Support Vector Machines (SVM), K-Neighbors, Gradient Boosting, Extra Trees and Random Forest. Among them Random Forests had shown maximum a median F1-score of 0.976. They also a Haar Cascade detector to classify the body part such as frontal face, right and left profile and torso. Skin pixel segmentation was done through a combination of several colors, texture, shape, spatial and unsupervised descriptors. Channel '*u' of the CIELuv color space turned the most informative feature for given purpose and the channel 'a*' of the CIELab model provedbest to enhance discrimination between acne lesion and normal skin. Further adaptivethresholdwas used on this channel to separate acne lesion from normal skin. Using the Laplacian of Gaussian filter acne spot detection and marking was achieved.The resultant was reports containing number, location and ray dimension of the detected acne spots. Also, in [16] authors used two-level k-meansclustering, modified version of "Color-simple k-means clustering.K-means clustering was applied to the LAB color space. At first levelclusteringtwo classes were defined such as skin and acne lesions and at the second level, three classes were assigned and one of desired class was acne. From level 1, desired cluster for executing the level 2 was chosen. Another method used was based on Texture analysis which attempts to quantify features such as rough, silky, smooth, or bumpy as a function of the spatial variation in pixel intensities. In acne images the background (skin) is smooth whereas the contours of acne exhibit more texture. Third method used the fuzzy c-means (FCM) method was utilized which allows a group of data to belong to two or more clusters. Classification task was achieved between acne scarring from inflammatory acne using SVM with linear kernel and fuzzy c-means method. In the classification task, it was divided mainly in three categories: comedones; inflammatory eruption-included papules and pustules, and severe eruptions that included cyst and nodules. K-means outperformed according to the authors.
In [17], Image blurring was done using Gaussian Blur to decrease noise in the image as they used region growing method whose drawback is noise. Also, the authors used pimples as the seed points on the face and once all seed point is placed then the region growing method is done. The results in a collection of pixels where each collection is called a region and these regions were further classified into one category of acne. For this, feature extraction of each region was done before selforganizing map and these were inserted in the normalized vector form into self-organizing map. At this stage, each region is grouped into a category. Acne types mainly contributing are papules, pustules, nodules and cysts and these can be segmented using region growing method. They had a note that the seed point is the darkest pixels of the acne. Also, threshold is adjustedand, in most cases, the ideal threshold value ranges from 60 to 70. They had various drawbacks in classification which will be addressed by them in later research. According to work done in [18], work is based on Markov random field (MRF) model [8] that proposed a new acne detection approach using associating chromophore descriptors. This was able to cope up with highlight and strong shading usually existing in RGB skin images. They did automatic acne detection for acne severity evaluation.

Figure-3 First row: original images. Second row: segmentation results using the MRF model and chromophore descriptors [18].
In the figure 3, inflammatory acne is outlined by the blue line, and hyper-pigmentation is outlined by the black line. The Professional evaluation of their algorithm in under at experienced dermatologist from a hospital (CHU Nice). This algorithm works best under images captured under uncontrolled environment. According to Chiun- Li Chin et. Al. [19], since the complete face cannot be used for acne assessment, they found fifteen action units from the input image. Once face is detected it is segmented into 11 face regions from earlier action units and these regions of face were used to detect skin condition as wrinkle, acnes and skin pore. They compared their result from the result of an experienced dermatologist judging the same image and their accuracy was 70%. The authors had conducted a multi-feature decision method to detect the skin condition on smart phones. They used Gabor, Laws mask and Kirsch feature to find the skin pore. Wrinkles were a linear feature on face and skin pore are circular feature. Hence after detection of acne area it can be separated into wrinkle and skin pore by feature shape. Their algorithm used Gabor filter, Laws mask and Kirsch filter to produce an image and connected-component labeling is done to it to analysis the information about every features.
In the research done in [20], they localized the acne as the first step before counting. They used template location on images for this purpose with chi-square test. This proved to be effective for counting acne lesions and can be used further for acne classification. In this technique similarity of overlapping blocks is determined using the blocks of the image with the template block.
In [21], authors not only detected acne but also tries to compress an acne image to maintain the accuracy in the data. The dataset used was from dermnet.com which is a public dataset. Compression was achieved using the wavelet transform by Haar wavelets. Further to detect the acne regions from the face image, K-means clustering was used with 3 clusters. Once the acne region is detected more compression was done on non-acne regions and less on acne region to maintain its accuracy. This paper had a drawback that some images having lip and hair were also segmented as acne region. Authors of [22] analysed that the proper diagnose of diseases is quite troublesome because of stiff hard-to discriminate nature of the symptoms they exhibit and hence Deep Neural Networks(DNN) [10], was used by the authors that outperform various state of art techniques of the time. DNN are really thriving today in Image Classification and Object and Pattern Discovery from images and videos. Convolutional Neural Networks (CNN) is a special DNN meant especially for computer vision and image classification problems. In this paper authors developed and used a Deep Residual Neural Network model for classifying five classes of Acnes from images. Their model achieved an accuracy of around 99.44% for one class and also above 94% for all other classes. The authors named their CNN model as "AcneNet" which is based on Deep Residual Neural Network [23]. "AcneNet" is a collection of many paths of differing lengths. They used small paths during the training and hence they do not solve invisible gradient problems by saving gradients across the entire depth of the network. The authors used 5 classes of Acne diseases in their experiment -Closed Comedo, Cystic, Keloidalis, Open Comedo and Pustular. The datasets required are huge for CNN and authors created it using images for 5classes of Acne diseases from dermnet.com. They used ADAM Optimizer [18] and used 80% of data for training and validation and 20% data used for testing.
In this research work [24], identification of acne type was made using Gray Level Co-Occurrence Matrix (GLCM) method and Support Vector Machine (SVM) algorithm. The authors evaluated that the shape and color of the acne image segmentation results, affect the value of the GLCM characteristics. The result had shown that acne identification using GLCMand SVM on a 256x256 image size gave accuracy of 89%, better than the image of 192x192 size that gave accuracy of only 66.7%. The maximum accuracy for testing data is 89% for 18 acne images.Image is sharpened to highlight an image intensity and image segmentation method used is Multi-Level Thresholding whose threshold value are taken in the YcbCr color space. GLCM is composed of location of adjacent pixels (d) and the angle between adjacent pixel locations and these results are used to calculate features as object texture representative. So using above mentioned methods, an infected face is automatically detected and classified as papules, pustules, and nodules or cysts. They did, following steps on the image as a processing step: • Image acquisition • Resize Image • Image Processing • Search GLCM value • Creating an acne group and save them in a format.mat • Creating a group of GLCM value and save them The applicationof the given research paper was created using MATLAB R2015b as the platform for acne type identification. Not only acne but some research shows significant work in skin disease cases as in [25].Various skin diseases can be ccharacterized by the pattern of skin infections which groups them in six non-covering groups such as healthy, acne, eczema, psoriasis, benign and malignant melanoma, etc. Whole research was accomplished in Four phases i.e., preprocessing, segmentation, feature extraction, and classification. Authors had used different digital image processing and ML techniques that gave a90 to 95% of accuracy using an SVM.CAD frameworks supported doctors to easily detect affected area as compared to other techniques. Various datasets National and International skin disease database were collected form the medical hospitals (Clinic) and research Centre from their respective institute websites but a number of diseases in them were not present in India. Hence the authors collected dataset by offline mode from regional database. Different size images with PNG and JPEG format with a 8-bit RGB color images were taken. During their study a total of 54 individuals will be screened. From these patients they took different skin disease images such as Eczema, Acne, Ulcer, vitiligo, Melanoma, skin code, WART, lap racy, Ringworm, and Phytophoto Dermatitis. Later on, after generating the dataset they implemented the proposed methodology. According to [26], as mobile images are increasing rapidly and world need to repair high-resolution face images automatically and quickly, they proposed an improved generative adversarial network (GAN)[27] method. A high-resolution dataset was used for training and testing with image size 256×256. next they used global average pooling layer to repair the moles and acne with varying shape and size. Later they used a mixed loss function for training. They achieved both qualitative results as well as quantitative results to be excellent. The authors first collected and made high-resolution human face image dataset (HRHF). GAN can only process 256 × 256 size photos whereas a high-resolution photo taken by mobile phones/HD digital cameras is very high. The authors solved the problem of repairing high-resolution images. Authors used mixed loss function to make network converge during the training phase, usingl1 and GAN loss. Their method gave significant results on the public dataset and HRHF dataset. Meanwhile, it can bring good subjective visual effect.GAN layer was used at the place of fully cconnected layer, so that the network can handle masks with arbitrary shape and size.

Dataset
As shown in the table 2, different authors use different datasets. Some of the researchers have used the free public dataset whereas some others have used regional datasets. Many others had generated their own dataset having images from highdefinition cameras or multispectral cameras. Moreover, in some cases the images were taken from the regional research centers and hospitals as done by [4] and [14]. Dermnet was used by most the researchers. In this section a description is provided for the public datasets that includes data of skin condition pictures. Currently, ISIC has maintained a record of multiple databases in which around 13,789 are dermatology images as on February 2019. The normal dermoscopic root is a well-structured and broad database. ISIC provides an online database on their portal [28]. There is another database named as PH2 database. This was developed at Pedro Hispaniola Hospital, Matosinhos, Portugal by the U.S. Research Center for Dermatology. This dataset have a collection of images with a resolution of 768*560 pixels with a collection of around 320 Dermoscopic images [29]. The dataset by dermnet can be found significant for preliminary work [30].

Conclusion and Future Scope
Acne is very common problem and hence in this review paper, methods associated with Acne detection, assessment, grading and classification are discussed. These are one of the most important processes in acne treatment and any automated system in this domain will help to make the process error free and with human interference.The first process of acne treatment is acne counting and classification which is manually done by dermatologist. Different methods were proposed in which concepts of Image processing, Machine learning and computer vision were used to automate the given task. In image processing most feature extraction was based on filtering, thresholding and clustering with k-means algorithm. In machine learning the most used classification methods used were SVM, CNN and LDF. Definitely the ML domain is bending towards the use of neural networks because of ease in accuracy. Many works concentrated on generating an automatic acne detection and classification system in which images are captured in real time use mobile phones or cameras.The most of the work done in this paradigm is done by Ramli et al. [5].
In most of the research paper the classification was limited to a maximum of five categories. This should be extended to more categories which we seek to be done in future. Also, the other concern with the given problem is the availability of common dataset that is internationally used. Human skin has a wide variety of color complexions and hence one system designed for a country may not be valid for the other countries and hence an extensive model should be designed to do the same so that it can be used worldwide. Further with the development of reinforcement learning models, a system can be designed at a hospital that can learn from the data itself obtained from the patients. This model will be trained such that it gives best accuracy. Acne assessment is a complicated task with a complete image because hair, eyes and lip regions doesn't make it possible to extract acne region of interest. Many authors tried to develop a real time application for the same but this can be in done future to make a mobile assisted acne grading model. In the BLOB model future work can be done by extracting features from each BLOB and comparing with set of acne features, to classify the detected BLOB is whether an acne or not.