A nondestructive method for estimating the total green leaf area of individual rice plants using multi-angle color images

Ni Jiang*, Wanneng Yang†‡, Lingfeng Duan*, Guoxing Chen, Wei Fang*, Lizhong Xiong and Qian Liu*¶ *Britton Chance Center for Biomedical Photonics Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology 1037 Luoyu Rd., Wuhan 430074, P. R. China National Key Laboratory of Crop Genetic Improvement and National Center of Plant Gene Research Huazhong Agricultural University Wuhan 430070, P. R. China College of Engineering Huazhong Agricultural University Wuhan 430070, P. R. China MOA Key Laboratory of Crop Ecophysiology and Farming System in the Middle Reaches of the Yangtze River College of Plant Science and Technology Huazhong Agricultural University Wuhan 430070, P. R. China ¶qianliu@mail.hust.edu.cn


Introduction
][4] Green leaf area (GLA) is closely related to plant growth, light interception, photosynthetic e±ciency, evapotranspiration and response to fertilizers and irrigation, 5 which basically determines the rice yield and quality. 6,7Therefore, it is a key parameter that needs to be accurately measured.
GLA can be obtained via the gravimetric technique after harvesting the leaves.The gravimetric technique is based on the leaf mass per area (LMA) determined from a sub sample, and the total leaf area is calculated based on the sub sample LMA and the dry weight of all leaves. 8Commercialized scanning planimeters, such as LI-3100 (LI-COR Biosciences, NE, USA), can make rapid GLA measurements for individual plants in an automatic but destructive manner.These destructive methods are accurate but labor-intensive.For nondestructive GLA estimation, special instruments have been developed.Plant canopy analyzers, such as LAI-2000 (LI-COR, Inc., Nebraska, USA) and the Delta-T Devices SunScan (Delta-T Devices, Cambridge, UK), are used for estimating leaf area index of ¯eldgrown rice plants.However, these instruments are not suitable for GLA estimation of individual plants.Portable scanning planimeters that support in vivo GLA assessment of individual plants, such as CI-202 (CID Inc., NW Camas, WA, USA), could be labor-intensive and time-consuming when measuring large quantities of leaves.Therefore, a nondestructive and high-throughput method for GLA estimation of rice is needed.
3][14][15] The visible light imaging technique has been widely used as a nondestructive method for GLA estimation.Baker et al. 16 showed that the total leaf area of a whole tree plant can be calculated from image-based areas.Leroy et al. 17 obtained single leaf measurements from digital pictures and reported relationships between various leaf measurements relative to the actual leaf area.Rajendran et al. 18 found a strong linear relationship between the total leaf area of individual wheat plants and the sum of the projected area in three orthogonal images.Marcon et al. 19 proposed two models for total leaf area estimations of co®ee plants using the width and the height of tree canopies obtained from images as well as the total leaf area visible on the images.Nagel et al. 20 also found a strong linear relationship between the total leaf area and the sum of two sideviews at a 90 horizontal rotation for maize and barley plants.Pereyra-Irujo et al. 21used a power function to describe the relationship between the total leaf area of individual soybean plants and the projected shoot area, which was calculated by summing one top-view projected leaf area and one side-view projected area from the images of the plants.However, nondestructive approaches are yet to be employed in estimating the GLA for individual rice plant.
Previously, our group developed an automated high-throughput system for measuring rice tillers (H-SMART 22 ).In this study, the visible light imaging system was incorporated into the H-SMART for rice image acquisition.In this paper, we present and validate a nondestructive method for estimating the total GLA of individual rice plants using multi-angle color images.The visible light imaging system was incorporated into the H-SMART for rice image acquisition.And the GLA was estimated through a modeling approach using projected areas of the plant in images.The objectives of this paper were: (i) to adapt the image-based method for estimating the GLA of individual rice plants; (ii) to propose a model using projected area for calculating the GLA; and (iii) to evaluate the accuracy improvement of GLA estimation by using multiangle color images.

Sample preparation and the destructive method
A total of 235 rice plants were used to develop and validate the models.The species of plants were Liangyoupeijiu and Peiliangyou 3076.All of the samples were imaged before the leaves were harvested.In this study, a commercial scanner (BenQ 8800, BenQ Corporation, China) was used to obtain the actual leaf area measurement.The destructive method was similar to that of Caldas et al. 23 Leaves were clipped and pasted °at on white sheets of paper, after which the papers were placed in the scanner.
The scan was performed with 300 dpi, and the images were saved as 8-bit grayscale images.Using a ¯xed grayscale threshold, leaves in the saved images were discriminated from the background.Based on the foreground pixel counts of all leaves that belonged to the same plant, the total leaf area for individual rice plant was obtained.The experiment was performed during the tillering stage.The minimum and maximum leaf area measurements were 30.34 and 1741.51cm 2 , respectively.

Digital image acquisition
When incorporated with visible light imaging system, the H-SMART was able to e®ectively acquire rice images.Figure 1 shows a schematic of the imaging system, which consisted of a side-view color camera (Stingray F-504C, Allied Vision

Image processing
The image processing procedure is illustrated in Fig. 2.
The variables r, g and b are the normalized color features calculated according to: where R, G and B are the grayscale values for each RGB channel.
A ¯xed threshold was applied to the E Â G image to identify bright objects, and a ¯xed threshold was applied to the E Â R image to identify dark objects.By applying an AND operation to the two binary image, the plant region was extracted, and the soil in the pot was not discriminated as plant [Fig.2(d)].Due to the presence of undesirable spots in the binary image, a processing step was employed to remove small areas, and regions with areas smaller than a prede¯ned threshold were eliminated [Fig.2(e)].
All of the 12 side-view images and one top-view image were processed in the same way.The projected areas of the plant in multiple views were calculated by counting the number of pixels that belonged to the plant region in each image.

Model development and validation
A total of 135 pots of the samples were randomly selected for model development, and the remaining 100 samples were used as test samples for model validation.Using projected areas of the plant in images, linear, quadratic, exponential and power regression models for estimating total GLA were developed based on regression analyses.For each model, the following statistics for assessing the goodness of ¯t were calculated: (1) Coe±cient of determination value (R 2 ): a measure of how well the model represented the data.
(2) t-test value and its signi¯cance: tests of individual independent variables indicated the statistical signi¯cance of respective independent variable.(3) Mean absolute percentage error (MAPE): a measure of accuracy, which was given as follows: where ELA i is the estimated leaf area using a model for the ith sample, and n is the number of plant samples.(4) Root mean square error (RMSE): a measure of error variance, which was given as follows: To assess the predictive performance of the models and test their ability to generalize, the test samples were used for model validation.The ELAs, which were determined using the models, were compared with the actual leaf area, and leave-oneout cross validation using the 235 samples was applied.The R  RMSEP and RMSECV), were calculated.Based on these analysis results, the best model was selected.

Performance of linear and nonlinear models
A total of 135 pots of rice plants were randomly selected as calibration samples for model development.To determine the best predictor of GLA, the relationship between the GLA and the top-projected area (TA) was compared with the relationship between the GLA and the average projected area of 12 side-view images (SA ave Þ.The scatter plots of TA vs GLA and SA ave vs GLA are shown in Figs.3(a) and 3(b), respectively.It can be observed that both TA and SA ave have a close relationship with GLA.
The two plots are similar, although the plot of TA vs GLA shows a greater dispersion of the data.Therefore, SA ave is the better predictor of GLA.
Using SA ave and GLA as independent and dependent variables, respectively, linear, quadratic, exponential and power models were developed.The models are: Exponential : GLA ¼ a 0 Â e a 1 ÂSA ave ; ðModel 3Þ Power : lnðGLAÞ where a 0 , a 1 and a 2 are the coe±cients.
To compare with the four models ¯tted to SA ave vs GLA, the image-based leaf sum (IBLS) model of Rajendran et al. 18 was also developed using the calibration samples.
where A sum is the sum of the projected area from the top-view image and two orthogonal side-view images.
Table 1 presents the regression coe±cients and statistical parameters for the ¯ve models described above.Figure 4 shows the relationship between the ELA using regression models and SA ave or A sum , as well as the dispersion pattern of the residuals for the respective models.As can be seen from Table 1, all of the models showed high R 2 c values, and the coe±cients were signi¯cant.For all of the models, the correlation R 2 p exceeded 0.87, and the accuracy on the test samples was slightly less than that on the calibration samples, and the estimation error predicted by the cross validation gave similar results to the error on the calibration samples.The results showed that the ¯ve models did not over¯t the calibration samples and had good ability to generalize.
In the ¯ve developed models, the exponential model had the worst precision.The scatter plot of the residual of the exponential model had a heteroscedastic behavior, and the estimation error was large when the plants were large [Fig.4 1); in addition, the residuals were close to a normal distribution.Model 1 and Model 5, where the relationship between the variables was linear, behaved similarly for GLA estimation.The nonlinear models, with the exception of the exponential model, ¯t the GLA better than the linear models.

Performance of the models using various numbers of side-view images
A greater number of side views would obviously have more information about the plant architecture.For example, additional hidden leaves can be observed when using multiple views.The projected areas of 10 randomly selected samples in each A nondestructive method for estimating the total GLA of individual rice plants 1550002-7 side-view image are shown in Fig. 5, where the curves of the projected area °uctuate, and the scale of this °uctuation varies among di®erent plants.
Since di®erent overlapping leaves were discerned in multi-side-view images, the projected area varied in di®erent side views.To determine the appropriate number of side views, power models that use the average projected area from di®erent numbers of side-view images for the same 135 samples were developed and evaluated: where SA 1 is the projected area from a single image with an angle of 0; SA 2 is the average projected area from two images with angles of 0 and 90; SA 4 is the average projected area from four images with angles of 0, 90, 180 and 270; and SA 6 is the average projected area from six images with angles of 0, 60, 120, 180, 240 and 300.Table 2 shows the results for the four power models.For all of the models, the ELA had a strong relationship with the actual leaf area.Comparing the predicted accuracy of the models, Model 6, which used a single side-view image, had the highest MAPEP value with 10.23%.In comparison, for Model 7, the MAPEP decreased by 4.12% points to          6.11%.In addition, the use of Model 8 or Model 9 further reduced the MAPEP to 5.99% and 5.87%, respectively.The same trend was observed in the RMSE values.Model 6 had the highest RMSE value, and the RMSE value dropped as the number of side-view images increased.These results showed that the use of multi-angle images was e±cient in reducing the estimation error, which was expected because the rice plants were not radially symmetrical with a straight axis of symmetry, resulting that a small number of views did not adequately represent the overall architecture of a plant.By comparison, Model 4 showed only a slight reduction in the MAPE and RMSE values relative to Model 8 and Model 9. Therefore, using more than four side-view images would give only marginal improvement.

Performance of the models with and without the top-view projected area
The use of a top-view image allowed for the correction of the leaf area estimation for the overlapping leaves in side-view images. 15Therefore, the value of the top-view image for leaf area estimation was tested in the models.The models were extended as follows: The parameters of the models and the error measures are listed in Table 3.The addition of the top-view image yielded better results than the use of the corresponding side-view images alone, which indicated that the top-view images was useful for correcting the estimates.In addition, Model 11 performed better than Model 10, which showed that adding TA as a separate predictor achieved greater improvement than incorporating it into the term (SA þ TA).The MAPEP value decreased by 1.85% points, to 8.38%, when using Model 11 instead of Model 6, whereas the MAPEP value decreased only 0.32% points when using Model 15 instead of Model 4. These results indicated that the improvement diminished as the number of side-view images increased, and the comparison of RMSE values for these models also indicated that adding the topview image did not signi¯cantly improve the performance of models, when using multi-side-view images.

Discussion
A nondestructive method for estimating the total GLA of individual rice plants was presented.Images of rice plants can be e®ectively acquired using the visible light imaging system, which was incorporated into the H-SMART system.9][20][21] In this paper, it is demonstrated that the GLA of individual rice plants also had a strong relationship with the plant area in the image, and the side-view projected area had a stronger relationship with the actual total leaf area than the TA.
Among the models mentioned above, the power models performed better than linear models, such as Model 1 and the IBLS model (Model 5).Model 6, which used only one side-view image, had accuracy similar to that obtained by the IBLS model, and the estimation error for Model 12, which also used three orthogonal images, was much less than that for the IBLS model.As shown in Tables 1 and 2, the coe±cients a 1 and a 2 in the quadratic model were positive, and the coe±cient a 1 in each power model was greater than 1.It can be inferred that GLA might have been allometrically scaled in terms of the projected area in the image.This scaling occurs because, the leaf overlapping becomes more extensive as the plant grew larger, resulting in an increase in the proportion of occluded leaves in the side-view images.
For each rice plant, 12 side-view images and one top-view image were taken.By comparing the performance of the models using various numbers of side-view images, it was found that the accuracy of the GLA estimation was increased as the number of side-view images increased.The reason was that rice plants lacked inherent asymmetry, resulting in varied projected area in di®erent view angles.To get the overall representation of plant area, it was necessary to use many di®erent view images.On the other hand, top-view image was useful for correcting the leaf occlusion.By comparing the performance of the models with and without the top-view projected area, it was found that estimation error decreased when TA was added.In addition, the improvement in accuracy was greater when adding TA as a separate independent variable instead of using the sum of SA and TA as the independent variables.This might be caused by the high correlation between TA and SA.However, the improvement provided by adding TA diminished as the number of side-view images increased.The reason was that the rice plants considered in the studies were erect growth forms and the leaves were tilted, thus multi-side-view images of the plant could correct the hidden leaf area and represent the total leaf area.In addition, the overhead camera distance should be long enough to avoid a di®erence in the pixel resolution between the top leaves and the bottom leaves; i.e., when the plants were excessively tall, it was di±cult to con¯gure the image acquisition system.Therefore, when attempting to estimate leaf area, the need to use a top-view image should be taken into consideration.And considering that the image acquisition and processing time increased according to the number of the images, the use of four or six side-view images should be su±cient to estimate the leaf area with high accuracy.
The power models proposed were able to estimate the GLA of individual rice plants with high R 2 and high precision.When using multi-angle color images, the R 2 p exceeded 0.98, and the MAPEP value was about 6%.The sources of error were mainly attributed to extensive leaf overlapping, nonuniform leaf orientation and nonuniform leaf density.In addition, the inevitable leaf sway during the rotation of the plant may have resulted in undesired discrimination of leaves, which also introduced error into the leaf area estimation.

Conclusion
In summary, we presented a nondestructive method for estimating the total GLA of individual rice plants.Models for estimating the total leaf area of rice plants were developed.Based on the projected area derived from multiple images, the power models were capable of estimating the total GLA in a nondestructive, accurate and e±cient manner.This method allowed time-lapse measurements during the plant growth period and, thus, the growth curve of plants could be attained.Using this method, researchers are able to monitor plant growth and control growth conditions in a timely fashion.The models in this study were designed for two similar rice varieties.In the future, certain key parameters characterizing di®erent rice phenotypes will be derived and incorporated in the model, and a model for various rice cultivars (e.g., rice germplasm resources) will be developed.
Technologies, Germany), a top-view color camera (DH-SV1410, Daheng Group Inc., China), a rotating platform, a servo motor controller (MBDDT-2210, Panasonic Corporation, China), a computer workstation (HP xw6400, Hewlett-Packard Development Company, USA) and light sources.Both cameras were equipped with an 8 mm focal length (fÞ lens (F1.8 EX DG Aspherical Macro, SIGMA, Japan), and the axes of the two cameras were oriented orthogonal to each other.The side-view camera was mounted rigidly with the viewing plane perpendicular to the ground plane at a distance of 1700 mm from the plant, and the top-view camera was mounted on a top plane at a distance of 1700 mm above the plant.To improve the image quality, a dark blue backdrop was used to provide a consistent background behind the plant.In addition, to prevent strong shadows, two light sources were ¯xed to the left and right of the plant respectively, and two light sources were ¯xed on the top plane.The cell size of the top-view camera was 6.45 m Â 6:45 m, and the picture size was 1392 Â 1040 pixels.The cell size of the side-view camera was 3.5 m Â 3:5 m, and the picture size was 2452 Â 2056 pixels.Therefore, the ¯eld of view for the top-view camera was approximately 1908 mm Â 1425 mm (length Â width), and the ¯eld of view for the side-view camera was approximately 1824 mm Â 1529 mm (height Â width).A total of 12 side-view images (angle-interval of 30 ) were acquired as the potted rice plant was continuously rotated, and one top-view image was taken.

Figure 2 (
a) presents a color plant image from the side view.To enhance the plant region extraction, the image was transformed to an excessive green (E Â G) image [Fig.2(b)] and an excessive red (E Â R) image [Fig.2(c)] using 24 :

Fig. 2 .
Fig. 2. Outline of the side-view image processing procedure.(a) The original image; (b) the E Â G image; (c) the E Â R image; (d) the binary image; and (e) the result of removing small particles.

Fig. 3 .
Fig. 3. Scatter plots of projected areas vs the actual total GLA.(a) top-view projected area (TA) vs GLA; (b) average side-view projected area (SA ave ) vs GLA.

Fig. 4 .
Fig. 4. Performance of the linear and nonlinear models.The relationships between the ELA and the average side-view projected area for modeling samples using the (a) linear model, (b) quadratic model, (c) exponential model, (d) power model and (e) IBLS model are shown in the left panel.The dispersion pattern of the residual for the respective (f) linear model, (g) quadratic model, (h) exponential model, (i) power model and (j) IBLS model are shown in the right panel.

Table 1 .
Statistical summary of the ¯ve developed models for total GLA estimation.
model gave the best performance and had the highest R 2 value, and highest accuracy (Table

Table 3 .
Statistical summary of the models with the top-view projected area.

Table 2 .
Statistical summary of the models using various numbers of side-view images.