Development of multiband optical sensing method for phenotyping of tomatoes in cultivation site

For the development of a smart food chain, it is necessary to produce stable high-quality agricultural products in the agricultural sector which is the starting point of the food system, and it is important to accumulate and analyse the phenotypic information of agricultural products during cultivation. An easy, rapid, non-destructive and quantitative evaluation method of tree vigor is desirable. This study aimed to develop an X-ray fluorescent (XRF) spectroscopy of fresh leaves and infrared (IR) spectroscopy of fruit juice using a portable device. In addition, the relationship between the spectroscopic information and the surface color of agricultural products was studied. The results showed that the changes in leaf element were in balance and organic matters in the fruits due to slight differences in cultivation conditions were grasped as the XRF and IR spectroscopic information. Furthermore, such changes could be reflected as the differences in the surface color information obtained using the digital camera. Therefore, it was experimentally suggested that the multiband optical sensing could be a powerful and important tool for realizing smart food chains and phenomics research starting from agricultural production.


Introduction
To develop an ecosystem for a smart food chain, it is necessary to stably produce high-quality agricultural products and to design the quality in agricultural sectors, which is the starting point of the food system. In particular, it is important to rapidly, non-destructively, and non-chemically acquire the phenotypic information of agricultural crops in a simple manner and to analyse them for cultivation management.
Plants absorb essential elements in the form of inorganic nutrients from their roots with water, and the balance of these essential elements in them plays a large role in their nutritional state. Additionally, various kinds of organic components in agricultural crops are very important not only as of the metabolic chemicals but also as the quality factor. Furthermore, the color of agricultural products has been practically and empirically used as one of the most important and useful quality indices in agricultural processes such as cultivation, harvesting, sorting, packing, logistics and retails. Therefore, future agriculture must equip various plant sensors to obtain such phenotypic information while the wireless sensor networks in the present stage mainly cover the weather and soil conditions (Kameoka and Hashimoto, 2013).
For such sensing in agricultural sectors, optical sensing techniques are desirable because of the potentialities for the rapid, easy, cheap, non-destructive, and non-chemical quality evaluation of agricultural products. Optical sensing could be characterized by comparison with separation analysis. In separation analysis represented by chromatography and electrophoresis, a mixture is basically fractionated to a single chemical specie. Conversely, in optical spectroscopic analysis, the optical spectroscopic information is collected as a mixture is acquired, the desired chemical species are identified from the spectrum information, and their contents are spectroscopically determined. The optical and spectroscopic methods could acquire extensive information collected by large-scale using various waveband in a short time.
We have studied the applications and possibilities of optical sensing techniques, from X-ray to terahertz radiation (Hashimoto and Kameoka, 2008;Kameoka and Hashimoto, 2015;Hashimoto et al., 2016) PAPER al. (2016) reported that the X-ray fluorescent (XRF) spectroscopy could be a quantitative method to simultaneously estimate the elemental contents by comparing the XRF spectroscopic information of the dried leaf powder with those of the extracts obtained using a conventional destructive method, and we also showed the great potentiality of the spectroscopic method to get the quantitative elemental information of tomato (Muramatsu et al., 2018). In addition, we successfully acquired the accurate infrared (IR) spectra of the simple cross sections of the petioles of tomato, and the experimental results suggested that the plural modes of nitrogen could be quantified base on the IR spectroscopic information . For other organic components in agricultural components, the potentialities of the IR spectroscopic methods have been reported (Kameoka et al., 1998;Kanou et al., 2005;Hashimoto and Kameoka, 2008;Kanou et al., 2017). Furthermore, we tried to develop a quantitative color evaluation method of agricultural products in the agricultural fields (Motonaga et al., 2004;. By analysing the relationships between the color appearances of the image taken in the agricultural fields using a commercial and cheap digital camera and the illuminating spectral information, we developed a quantitative color appearance evaluation method of agricultural product images (Hashimoto, Suehara and Kameoka, 2015;Hashimoto et al., 2015;Hashimoto et al., 2017).
In this study, we focus our attention on the XRF spectroscopy, the IR spectroscopy and color images analysis, which could respectively acquire the plural chemical elements at the same time, the organic substances such as sugars and organic acids, and the appearance as the comprehensive phenotype. The purpose was to develop the optical sensing methods in consideration of the applications in the agricultural fields and to grasp the relationship between the optical sensing information for evaluating the tree vigor.

Materials
Tomato (Lycopersicon esculentum cv. Moneymaker) was selected as the plant material. The tomato plants were hydroponically cultivated in a greenhouse at Togo Field, Field Science Center, Nagoya University, Togo, Japan. After cultivation at 56 days under the standard conditions using the usual culture medium, the modified culture medium, which is deficient in nitrogen (N), phosphorus (P) or potassium (K), was supplied only for the last 2 days in order to change the vigor.
In addition, tomato (Lycopersicon esculentum cv. Chukanbohon Nou 11) was used as plant material for the developments of the surface color measurements and of the analysis on the relationship between the surface color information and the inner quality information. The tomato plants under experiment were cultivated in an artificial intelligent growth chamber (Aiterrarium, Matsushita Electric Works, Ltd.) controlled via the Internet using vermiculite as grown soil. The tomatoes were stored through the post-ripening process in a growth chamber controlled at 298 K and relative humidity of 75 % using saturated NaCl aqueous solution.

XRF spectral measurement
The tomato compound leaves just under the fruit bunch were sampled and were stored in a sealed container kept at 277 K. Each fresh leaflet was used as the sample for the XRF spectral measurement. The dried leaflet powders were also prepared for the XRF spectral measurement as the conventional method. After all leaflets on the same compound leaf were dried in an oven at 353 K for 72 hrs, the ground powders passed through a 600 μm sieve were mixed for the spectral measurement.
An Innov-X DELTA Premium Handheld XRF Analyzer (Olympus Corp., Tokyo, Japan) was used for the XRF spectral measurement by making a special measurement station. The XRF spectra of each fresh leaflet were collected on three positions by avoiding the leaf vein. The measurements were made three times at each position. The XRF analysis was performed on 7 leaflets on each compound leaf. Additionally, the XRF spectra of the mixed dry powders for each compound leaf were acquired.

Infrared Spectral Measurement
A Fourier Transform infrared spectrometer (FTIR; Spectrum 400, PerkinElmer) with an attenuated total reflection (ATR) accessory (9 reflections, Dura Sample IR, Applied System) was used for correcting the IR absorption spectra of the tomato juice squeezed from the whole tomato fruit. A total of 64 scans of symmetrical interferograms at 4 cm -1 resolution were added for each spectrum. The instrument was allowed to purge for several minutes prior to the acquisition of spectra in order to minimize the spectral contribution due to atmospheric water vapor.

Color image acquisition
The standard RGB color charts with plain red, green and blue sections (Japan Color Research Institute) for the color calibration under the natural lighting conditions   prepared. The setup of the color image acquisition system was schematically the same as that used in our previous studies (Motonaga et al., 2004). The system is composed of a digital camera (AIR A01, Olympus Corporation), six fluorescent lamps (TRUE-LITE, DURO-TEST Co., Ltd.) (Marutoki Ltd., 2016) and two white diffuse reflectors (440 x 440 mm). The use of the TRUE-LITE fluorescent light at a 5227°C (5500 K) color temperature is based on the CIE (Commission Internationale de l'Eclairage) regulations. The sample was set just under the camera and could be located almost in the center of the image view. The acquired images were recorded as RAW format files. The visible spectral radiance was measured using the homemade spectroscope equipped with an ultra-compact minispectrometer (C10988MA-01, Hamamatsu Photonics K.K.), which has a thumb-sized (27.6×13×16.8 mm) spectrometer head for installation into the mobile measurement equipment and uses a CMOS image sensor integrated with a light-receiving slit . On the other hand, the images were also taken under the field conditions facing the window set to the southwest direction on the 6th floor of the building of Graduate School of Bioresources, Mie University, Tsu, Japan (north latitude 34.745958333, east longitude 136.522975000). For the image acquisition under the natural lighting conditions, the aperture priority automatic exposure (AE) mode, at which the F-number was fixed at 8, was applied to the open platform camera and the shutter speed was set at "Automatic". The sample images were extracted from the acquired ones using Photoshop R CS6 (Adobe). We obtained the RGB values in the RGB color space for each pixel of the extracted image and calculated the average values for the color parameter evaluation.

Results and discussion
3.1 X-ray fluorescent spectroscopic characteristics of flesh leaves Figure 1 displays the examples of the XRF spectra of the fresh leaflet and the dried powder. The spectra were standardized using the peak intensity of RhLα that was the target for X-ray generation and were averaged for the seven leaflets of the same compound leaves. For both kinds of the XRF spectra, the excellent agreements of the peak positions relating major elements were observed though the standardized intensities for the dried powder were much stronger than those for the fresh leaves. As shown in Figure 1, the elemental balance conditions in the fresh leaves could be reflected by the XRF spectral information and the intensity differences could be influenced by the differences of the water contents between the spectral measurement samples. Figure 2 shows the comparison of the peak intensities of the standardized XRF spectra of the fresh leaves of tomato cultivated under the K-deficient conditions for two days with those cultivated under the standard conditions. Good agreements were observed between the two kinds of the standardized intensities for Ca, S, P and Fe. However, for K, the intensity for the fresh leaves of tomato cultivated under the K-deficient conditions was slightly lower than that cultivated under the standard conditions. Hence, the results represented in Figure 2 suggested that the XRF spectral information obtained using the developed method (Muramatsu et al., 2018) could be effective for the evaluation of the elemental balances in the fresh leaves.

Infrared spectroscopic characteristics of tomato juice
Figure 3(a) shows the second derivative spectrum of the tomato fruit juice, and the absorption peaks relating to sugars, amino acids and organic acids that could characterize the quality of tomato fruits were observed. As an example, Figure 3 Figure 3(b), some peaks of the subtracted spectrum were observed under the gray band, and it suggested that glucose and amino acid contents in the juice increased by performing the Kdeficiency operation. In addition, the influences of the other elemental deficiency treatments on the IR spectral features were similarly observed. It was then thinkable that the IR spectral analysis could be effective for the evaluation of the organic component differences in the tomato fruit juice.
As described above, the differences in cultivation conditions were regarded as the differences in the optical spectral information. Therefore, the combination of the XRF and IR spectroscopic analytical methods could be applicable to understanding the vigor and to the cultivation control based the optical spectroscopic information.

Relationship between surface color and spectroscopic characteristics
The color of agricultural products has been practically and empirically used as one of the most important and useful quality indices in agricultural processes such as cultivation, harvesting, sorting, packing, logistics and retails. The color information could be reflected by the pigment components, which are some of the final productions through the primary and secondary metabolic process, and by the geometrical structure such as the shape and surface conditions. In addition, the objective and quantitative color evaluation method are then significantly desired in a wide variety of agricultural fields.
We have studied a quantitative surface color evaluation method of agricultural products under natural lighting conditions using a digital image acquisition system with geometrical flexibility between the camera system and the agricultural samples Kameoka, 2012, 2015;Hashimoto et al., 2015;Hashimoto et al., 2017). In consideration of the color calibration using the color images taken by the cheap and popular digital camera, the influences of the image recording format on the color calibration results were experimentally studied. By comparing the color parameters of the tomato fruit images calibrated using the color parameters of the virtual color chart simulated based on the illuminating spectral information with those using the color parameters of the actual color chart image taken under the natural lighting conditions, it was found that the former method could be less susceptible to the image compression. Figure 4 displays the time courses of the H values of the tomato fruit images during post ripening at 25 °C. The calibrated values consistently agreed with the standard values, and the influences of the file format (image file compression) were very slight. These results could be attributed to the former method did not use the actual image for the simulation of the color parameters of the virtual standard color chart.
The development of a color measurement system compatible with agricultural sites was attempted. The XRF and IR measurements were performed on tomatoes cultivated with using the culture media with the different nitrogen concentrations, and the principal component analysis (PCA) was performed using P-, K-, Ca-, and Fe-XRF spectroscopic information of the leaves related to nitrogen in plants and the IR spectroscopic information of fruit sugars and glutamic acid. Figure 5 represents the PCA results, and the differences in fruit quality based on  Since the influences of the nitrogen concentrations in the cultivation media on the tomato vigor could be grasped as the XRF and IR spectroscopic information as shown in Figure 5, we studied the relationship between the color information and the inner quality information. As a result, the changes in some chemical elements in the leaves and organic components in the fruits could reflect the change in the hue values. Figure 6 shows the example of the relationship between the hue values of the tomato fruit surfaces and the second derivative values of the juice at the wavenumbers relating to fructose and glutamic acid. These results could suggest the possibility of easily cultivating and controlling the quality of agricultural products based on color information.

Conclusion
The XRF and IR spectral information of the tomato fresh leaves and the fruit juice could respectively reflect the elemental contents and the organic components such as sugars and amino acids. Additionally, the quantitative surface color evaluation method of agricultural products in consideration of the influences of the image acquisition conditions on the color calibration effect was developed. Consequently, this study plays a very important role in the development simple, non-chemical, non-destructive, simultaneous, and rapid method for understanding the phenotypes of plants during cultivation based on the multiband optical sensing information. Furthermore, the developed method could be an important and useful tool for realizing smart food chains starting from agricultural production and phenomics research in conjunction with advanced information technologies.
Applications of optical radiation to advanced sensing in plant phenomics and foodomics. Journal of the Illuminating Engineering Institute of Japan, 100(11), 497-501.