Development of smartphone-based imaging techniques for the estimation of chlorophyll content in lettuce leaves

Leaf color is a good indicator of plant’s health status. In this study, a new image acquisition technique was developed to estimate chlorophyll content of lettuce leaves. The images of lettuce leaves grown under artificial light were acquired using a smartphone. Leaves images was captured by directly attached the leaves to the camera lens with the aid of background illumination from SMD LED. Red, green, blue (RGB) color indices were extracted from leaves color images and some vegetation indices were also calculated. Then, the correlation between these indices and chlorophyll content obtained from SPAD502 chlorophyll meter were evaluated. Significant correlation was found between all the image indices and chlorophyll content with the R ranging from 0.63 to 0.85 except for G and B indices from RGB component. Highly significant correlation was found between vegetation indices (VI) and chlorophyll content (R = 0.85) with the lowest root mean square error (RMSE) of 8.07 g of chlorophyll/100 g fresh tissue. This demonstrated that the chlorophyll content of lettuce leaves can be successfully estimated using regular smartphone with added background light illumination from SMD LED.


Introduction
Leaf color is often used to indicate the health status or growth of the plant. Commonly, a healthy plant is characterized by the greenness of the leaf (Friedman et al., 2016). Generally, the green pigment in the leaf is caused by chlorophyll where it absorbs red and blue wavelength from the solar radiation while leaving green wavelength to be reflected which make a leaf to appear green. Chlorophyll is important in photosynthesis process because it allows the plant to absorb energy from sunlight. Thus, the estimation of chlorophyll content of plants provides useful information on the plant growth which pertinent to the crop management.
Conventional foliar tissue analysis using organic extraction has been used as a reference method to determine chlorophyll content (Jiang et al., 2017). This method involves the extraction of chlorophyll from leaves samples which requires mechanical separation and dissolution process using organic solvents like acetone (León et al., 2007). Although this method is the most accurate measurement for chlorophyll content, the procedure involved is destructive, laborious, timeconsuming, complex and expensive (Ramirez, 2010). To overcome these shortcomings, researchers had investigated and developed various indirect methods to estimate chlorophyll content.
One of the indirect measurements is by using chlorophyll meter. This device provides non-destructive, portable and rapid measurement to estimate chlorophyll content (Wu et al., 2007). The most reliable chlorophyll meter is handheld SPAD-502; which has been widely used by agricultural researchers over the past few decades (Ali et al., 2012). Handheld SPAD meter was developed by Minolta Co. (Japan) (Lin et al., 2010) and it measures the transmission of red and infrared (650 nm and 940 nm, respectively) light through the leaf. Despite their potential as a reliable non-destructive approach that can replace conventional method, this device is relatively expensive and often produce high variability in readings due to small measurement area of only 3x2 mm 2 (Li et al., 2009;Lin et al., 2010).
Apart from chlorophyll meter, digital imaging technique also has been investigated to estimate the nutrient status of plants in recent years (Yadav et al., 2010;Tewari et al., 2013;Rigon et al., 2016;Vesali et al., 2017). Since chlorophyll content affects the leaf color, digital imaging based on red, green, blue (RGB) color model is often used to measure chlorophyll content. This technique provides a rapid diagnosis by using a low-cost instrument like a digital camera (Muñoz  -Huerta et al., 2013). However, the effect of environmental light such as shadow or sunny condition on the image is the main concern when using this technique (Vesali et al., 2017). To solve this problem, researchers had developed several approaches to acquire consistent leaves images. Mercado-Luna et al. (2010) built a set up to capture tomato leaves images by using 7.2 mega pixel charged-coupled device (CCD) camera and fluorescence lamp (100W) to give constant illumination with adjustable camera height. The leaves samples were placed on the white background and the images were taken in a dark room. Another method of image acquisition developed by Tewari et al. (2013) using 5.1 mega pixel CCD camera and four fluorescence tube light to control illumination. The camera position was fixed inside a covered trolley.
Recently, Vesali et al. (2015) introduced contact imaging technique using smartphone with CCD camera sensor to determine chlorophyll content of corn leaves where the leaves image were taken by placing the leaves sample in front of the camera lens to diminish the effect of environment light on the image. In addition, this technique does not need a special set up and it eliminates the need for a pre-processing procedure like background segmentation since the camera only captured the leaves color. Later in 2017, they compared the image captured using method they used in 2015 with the image captured with background light from LED. They found an increase of R 2 from 0.42 to 0.97 for the correlation between vegetation index (VI) and chlorophyll content.
To the extent of our knowledge, most of previous work discussed above utilize digital camera equipped with CCD image sensor. The CCD image sensor has long been used in image sensor technology and it produce high quality of image with low noise (Gamal and Eltoukhy, 2005). Despite their advantages, CCD image sensor is power consuming and expensive (Tim, 2014). As replacement to CCD image sensor, complementary metal-oxide-semiconductor (CMOS) image sensor was widely used in smartphone camera sensor (Tim, 2014). This image sensor have smaller feature than CCD which reduced the cost of fabrication and make it less expensive with low power consumption (Tim, 2014). In this study, a method of image acquisition using smartphone camera which consist of CMOS image sensor with the aid of SMD LED as background illumination was introduced to reduce variability on the captured images. The relationship between chlorophyll content and indices extracted from the leaves images taken from the smartphone camera were then evaluated.

Plant material and experimental set up
Loose leaf lettuce (Lactuca sativa L.) was cultivated in hydroponic culture solution at the Biosystems and Environment Laboratory, Universiti Putra Malaysia. Seeds were sown in 44 mm peat pellet for 5 days. Then after germination, seedlings were grown for 4 weeks under different ratio of red (R, 620-660nm) and blue (B, 450-470nm) light emitting diode (LED) grow light. Two spectral treatments were used in this study which include ratio of 5B:1R and 5R:1B. The LED lights were supplied from Malaysia Hydroponics (Thunder Ray Sdn. Bhd., Selangor, Malaysia) which consisted of 1.78 m linear fixtures with an array of 72 LEDs. The photoperiod was 16/8 h (day/night) and photosynthetic photon flux density (PPFD) was 150 μmol/m 2 ·s (day/night).

Image acquisition
Method of image acquisition was introduced using an android smartphone which equipped with CMOS image sensor and SMD LED background light illumination to provide a constant brightness for all the images captured and reduce the effect of ambient lighting condition (Figure 1). In this technique, the leaf was directly attached to the camera lens on top of the SMD LED and the camera captured the light passing through the leaf (Figure 1). Samsung galaxy J2 smartphone (Samsung Engineering Co., Ltd, Seoul, Korea) was used to acquire 30 images of lettuce leaves grown under each ratio of blue and red LED grow light. All the images collected were in RGB color space.

Chlorophyll content measurement using SPAD-502
Chlorophyll content were measured using SPAD-502 (Minolta Co. Ltd., Osaka, Japan) at the same leaf position where image had been captured using the smartphone camera. The SPAD device records the ratio of red and infrared light transmission called SPAD value where Y is total chlorophyll content in (g of chlorophyll/100 g fresh tissue) and X is the SPAD value measured by the SPAD-502. The measuring accuracy of SPAD-502 is within ±1.0 unit for a range of 0 to 50 under normal conditions (Dong et al., 2019).

Image analysis
Each image was captured in 8 bit RGB mode and saved in JPEG format. The RGB component were extracted from each images using Image Processing Toolbox of Matlab R2014a software (The Mathworks, Inc.,Natick, MA, USA) and then used to investigate their correlation with chlorophyll content in lettuce leaves. Then, three other combined indices from previous studies; GMR (difference between the green component and red component), GDR (green divided by red) and VI ((G-R)/ (G+R)) were also calculated (Wang et al., 2013;Wang et al., 2014). The mean of all indices including the RGB color space and other combined indices were correlated with Chl content obtained using SPAD meter according to Equation 1.

Statistical analysis
Experimental data were statistically analyzed using SAS 9.1 (SAS Institute, Inc., Cary, NC, USA). Analysis of variance (ANOVA) and Duncan's multiple range test (DMRT) were carried out to investigate the effect of different treatment of LED grow light on the SPAD value, chlorophyll content and image color indices at 5% probability level. Regression analysis were conducted using Microsoft Excel 2013 (MS Corporation, Redmond, WA, USA) to investigate the relationship between image indices and chlorophyll content obtained using SPAD meter. The data were randomly split into training and validation set where training set was used for developing the regression equation models, while the validation set was used to evaluate the developed models by calculating the root mean square error (RMSE): where N is the number of samples in the dataset, ŷ i is the predicted chlorophyll value calculated using the regression equation and y i is the chlorophyll measurement obtained through experimental procedures using the validation dataset.

Variation in SPAD values and chlorophyll content of lettuce leaves grown under different LED grow light
Result from ANOVA showed that there were significant differences (P < 0.0001) in SPAD value and chlorophyll content of lettuce leaves grown under different ratio of blue and red LED (Table 1). To further investigate the differences, DMRT was performed to compare the means of the SPAD values and chlorophyll content of lettuce. The mean SPAD value and chlorophyll content for lettuce grown in 5B:1R is slightly higher (31.82 and 127.19 g of chlorophyll/100 g fresh tissue, respectively) compare to 5R:1B (24.91 and 103.69 g of chlorophyll/100 g fresh tissue, respectively) ( Table 1). This was expected because blue light promotes the production of chlorophyll in plant leaves (Banaś et al., 2012;Son and Oh, 2013).

Relationship between image indices with chlorophyll content
All indices demonstrated significant correlation with chlorophyll content with the coefficient determination, R 2 ranging from 0.63 to 0.85 except for G and B indices from RGB component (Figure 2). The G and B indices showed very low correlation with the chlorophyll content and this can be seen by the R 2 of 0.12 and 0.15, respectively. This findings are similar to the results presented by Vesali et al. (2017) where they found B and G indices demonstrated very low correlation with chlorophyll content. While, R indices have significant relation with chlorophyll content. There was negative linear correlation found between R indices and chlorophyll content with the R 2 of 0.63 (Figure 2a). Their relationship can be expressed in the regression equation as shown in Table 2 and the model had RMSE of 9.63 g of chlorophyll/100 g fresh tissue. Although G indices poorly fitted the chlorophyll content, the combination of R and G indices such as GMR, GDR and VI denoted highly significant correlation with the chlorophyll content with the average of R 2 = 0.85 (Figure 2). Among these indices, VI index gave the lowest RMSE (8.07 g of chlorophyll/100 g fresh tissue) for the chlorophyll prediction. This showed that from our experimental data, the chlorophyll content is possible to be predicted up to (

Conclusion
In this study, the method of image acquisition using a CMOS image sensor on an android smartphone camera with the aid of SMD LED as background illumination was introduced. Various image indices were extracted and calculated from the RGB color space. These indices were then correlated with chlorophyll content obtained from SPAD meter. The image indices of R, GMR, GDR and VI were found closely related to the chlorophyll content and VI was found to have strongest linear relationship with chlorophyll content (R 2 = 0.85) with lowest RMSE (8.07 g of chlorophyll/100 g of fresh tissue). These results demonstrated that the chlorophyll content of lettuce leaves can successfully be estimated using regular smartphone that equipped with CMOS image sensor with added background light illumination from SMD LED. It could be used for the development of portable image acquisition tool for assessment of chlorophyll or nitrogen content in plant.