Shadow Identification in Food Images using Extreme Learning Machine

Shadow identification is important for food images. Different applications require an accurate shadow identification or removal. A shadow varies from one image to another based on different factors such as lighting, colors, shape of objects, and their arrangement. This makes shadow identification complex problem and lacking systematic approach. Machine learning has high potential to be used for shadow recognition if it is used to train algorithms on wide number of scenarios. In this article, Extreme Learning Machine (ELM) has been used to identify shadow in shadow mask area. This shadow mask area was determined in the image based on edge detection, and morphological operations. ELM has been compared with Support Vector Machine (SVM) for shadow identification and shown better performance. Keywords—Extreme machine learning; shadow identification; food images; support vector machine, edge detection; color spaces


INTRODUCTION
Shadow is defined as a result light blocking from its sources by an object, which causes the shadow to appear in another object.The shadow appears in the area that does not receive the light directly from its source [1], [2].Removing shadows from image is effective in simplifying the tasks of image processing and computer vision algorithms.However, removing the shadow has to maintain the information in the original image and the other details except the shadow [3].
In various applications, shadow provides important information regarding the scene, which as a result makes it useful to preserve the shadow information.However, in certain type of applications such as object recognition based on shape and color shadows becomes disturbing factor when they interfere with the object, which makes crucial to develop shadow identification and removal algorithm.This is why shadow identification and removal is considered to be essential [4], [5].Examples of the applications that require shadow identification and removal are segmentation, scene analysis, tracking, and object detection [6].
Machine learning is a fast emerging field and is receiving high recognition from the researchers as an effective tool in wide range of applications that involve data analysis, learning from high amount of data or features [7], [8].Computer vision is one important application of machine learning and it has been applied in image segmentation [9], [10], vision based learning, autonomous car [11].The appealing aspect of machine learning is providing the machine with the capability of learning from scenarios using implicit mathematical models where there is difficulty in providing explicit mathematical models due to the unbounded number of cases with the high complexity levels.One of the examples of such applications is the phenomenon of shadows.Yet shadows appear because of lighting conditions, there are unlimited scenarios of shadows shape, distribution, variations, and there is no explicit rules to create a boundary between pixels that pertain to shadows and others that belong to the original color of the object [12].This argument creates a motivation to develop machine-learning model for identifying shadows using plenty of training scenarios.To the best of our knowledge, this article is the first in developing and applying shadow identification algorithm based on extreme leaning machine that is one of the most efficient and recognized learning algorithms in the literature.
The organization of the article is as follows.The next section is introduced related work.In Section III, materials and proposed method are provided.Results and discussion are given in Section IV.Finally, conclusion and future work are provided in Section V.

II. RELATED WORK
The literature contains different shadow identification approaches.From a taxonomy perspective, shadow-identifying approaches are categorized under two classes: model based and property based.The former detects the shadow based on predefined geometrical, or illumination shape, while the latter describes the shadow based on features such as geometry, brightness, or color [1].Some shadow removal work requires no prior knowledge regarding the scene.Levine and Bhattacharyya [13] used boundary information to identify shadow regions in the image based on support vector machine (SVM) and then assign them the color of non-shadow neighbors of the same material.Learning based approaches for removal of shadows have been used in challenging type of applications such as identifying shadows in monochromatic images where the authors [14] have used a Boosted Decision Tree integrated into a Conditional Random Field (CRF) based model to identify the shadow based on extracted features: www.ijacsa.thesai.orgshadow variant features, shadow invariant features, and near black features.Regardless the useful results of this work, it focuses on challenges of monochromatic images, which is not used anymore in recent devices.Other researchers have identified shadow based on region.Guo, Dai and Hoiem [5] have predicted relative illumination conditions between segmented regions from their appearance and performed pairwise classification based on such information.This work is based on an assumption that all surfaces that contain shadows are planar and parallel to each other, which is not met in call cases.Also, their shadow detection might fail in case of multiple light sources.Some researchers have incorporated near infrared features with color information to define the shadow based on the framework of [15].In [3] the authors have developed a method to remove the shadows from real images based on probability shadow map.The probability shadow map identifies the amount of shadow that is affecting the surface.
Identifying shadows has been tackled in different applications, determining shadows in food type of images is among them.Patel, Jain and Joshi [16] have aimed at locating the fruit on the tree for harvesting purposes.Removing shadow is important to easy the segmentation process.The authors have applied Gaussian filter to remove shadows, which is not effective.Other researchers have applied shadow removing for specific food items such as banana as the work in [17] which shadows were reduced by arranging the distribution of illumination but no full removal of shadow was accomplished.
Unsupervised machine learning has been used also in shadow identification.In [12] the characteristic of the derivative difference of the brightness and light invariant have been used to automatically cluster pixels to generate shadow mask.This approach has been used to solve the shortcoming of the work in [18], however it does not work for vague shadow boundary.
After reviewing the previous approaches, it can be concluded that most of the shadow identification works are based on simplifying assumptions or easy testing scenarios where the shadow is created because of single light source or the object is single or non-connected to nearby objects.Building a more practical shadow identifier requires developing trained model based on effective machine learning.

III. MATERIALS AND PROPOSED METHOD
The experiments of this article have been done on dataset combined of 300 images.In this dataset, images are acquired by smartphone with 8 mega pixels with different lighting conditions; each image contains fork, knife and plate within food.
In order to detect the shadow in the image, a supervised model has been built based on Extreme Learning Machine (ELM).ELM was used for three reasons: firstly it is a supersized learning approach, which enables us to train the method based on examples of shadows.Secondly, this approach is effective in avoidance of local minima.Thirdly, this approach does not suffer from over-fitting similar to other supervised approaches like SVM.In the next subsections, the following points are presented.Firstly, the extracted features are given in subsection A. Secondly, the training of the model is provided in subsection B. Thirdly, running the classifier of the shadow identifier is presented in C.

A. Feature Extraction
The image was decomposed into set of blocks or windows that have to be classified as shadow or shadow free blocks.In order to do so, set of features has been extracted from each block.Statistical features are extracted such as mean: average or mean value.For a random variable vector a made up of N scalar observations, the mean is defined as: ∑ Also, the variance for a random variable vector a made up of N scalar observations, the variance is defined as ∑ Where, μ is the mean of A.
Also, the skewness has been extracted which is a measure of the asymmetry of the data around the sample mean.The skewness of a distribution is defined as: Where is the mean of , is the standard deviation of , and represents the expected value of the quantity .Skewness computes a sample version of this population value.
Besides the statistical features, the windows are converted into different color spaces, and also calculate descriptive statistics for them too.Color spaces are: YUV, HSV, II2I3, YCbCr, La*b* and Gray scale.

B. Training the ELM Model
Dataset has been built from wide range of images contain different arrangement of food items with some shadows in some parts.Total size of the dataset is 800 records.Half of the records (400 is chosen) from the dataset are used as a training data, and the other half (400) are used records are as testing data.This percentage of decomposition has been chosen because it is the most suitable one for avoiding over-fitting.Brute force approach has been used to find out the best number of neurons in the ELM for better performance.The best testing accuracy was accomplished for sigmoid function and for 50 neurons.

C. Shadow Identifier
In order to identify the shadow in the image, the typical approach is to apply the trained ELM on all the image blocks.However, this will result in identifying shadows inside the item of food.This might lead to removing parts inside the food item and will cause degradability in the shape of the item.The other way is to identify the shadow at the borders between the food items, which is useful to avoid over and under segmentation and to maintain the quality of the shape of the food item.The approach for identifying the shadows at the borders was by building the shadow mask.The shadow mask can be defined as www.ijacsa.thesai.org the area in the image in which the ELM shadow detector will be applied.As shown in the pseudo-code of Fig. 1, the procedure is combined of sequence of steps.Firstly, the Region Of Interest (ROI) is extracted to represent the actual food items in which the algorithm will be applied.Next, edge detection (Prewitt) is applied to extract the borders of the item.This approach has been used because it is gradient based easy to implement comparing with other edge detection.The only problem of Prewitt is its sensitivity to noise, which is not an issue in this stage as the processed image is simply a binary image and is not subject to noise comparing with an original raw image.
These borders are the region in which the morphological operations are applied for performing thickness and closing in order to add nearby area to the border and to maintain continuity.The result represents the mask that is provided to ELM shadow to identify in which windows shadows exist.Pseudo code of the proposed method.

IV. EXPERIMENTAL RESULTS AND DISCUSSION
For evaluating the developed approach of shadow detection, different types of food images were used.Fig. 2 shows an image with two food items; all the intermediate steps are shown.The shadow mask was not applied on the whole image, instead it has been applied only on a part of the image where the shadow is expected to appear on the surrounding part of the items as it is shown on Fig. 3.This region is used for testing the shadow algorithm to identify the shadow part and the non-shadow part.
Results of shadow identification were generated for different sizes, colors and number of food items with the developed approach as it is shown in Table 1.The algorithm was able to identify the shadow (red color) and the non-shadow (blue color).It is observed that some parts were identified falsely as shadows.However, this does not degrade the performance because it happens only on the surrounding part of the item and it can only lead to removing small parts in the borders as what it happens in item 3.For further evaluation, ELM classification has been compared with SVM and visual results are shown in Fig. 4. Obviously, ELM was better in identifying shadows than SVM which has failed in some parts in the borders and led to non-smooth results of shadow removing.

V. CONCLUSION AND FUTURE WORK
Shadow was identified based on combination between shadow mask approach and machine learning approach.For shadow mask edge detection and morphological operations were used while for machine learning ELM was used.The developed approach has been evaluated on different number, shape, and color of food items with different lighting and arrangement of food items.Results have shown good performance.ELM has been compared with SVM and results of ELM have outperformed SVM.Future work is to evaluate this approach as a part of object recognition approaches such as (b) (a) www.ijacsa.thesai.orgitems identification and calories estimation.In addition to that, the developed approach has to consider adding more features for shadows identification.This might have a role in increasing the accuracy of shadow identification and decreasing the rate of positive false.

Fig. 2 .Fig. 3 .
Fig. 2.An example to shows steps of the proposed method (a) Original image (b) Edge detection (Prewitt), thickness and close morphological operation.
Fig. 4. (a-b) Comparison detection and removing shadow between proposed (i) raw image (ii) detection shadow by SVM and (iii) detection shadow by ELM.

TABLE I .
RESULTS IDENTIFYING SHADOWS OF FOOD ITEMS