Development of a machine vision system for rice seed inspection system

Ruslan, R., Khairunniza-Bejo, S., Rukunudin, I.H, Jahari, M. and Ibrahim, M.F. Department of Biological and Agricultural Engineering, Universiti Putra Malaysia, UPM 43400 Serdang, Selangor Malaysia School of Bioprocess Engineering, Universiti Malaysia Perlis, UniMAP 02600 Arau, Perlis Malaysia Center of Excellence for Advanced Sensor Technology (CEASTech) Universiti Malaysia Perlis, UniMAP 02600 Arau, Perlis Malaysia


Introduction
Machine vision technology has been utilized in several applications in the agricultural sectors, such as land-based and aerial-based remote sensing for the application in precision agriculture, natural resources assessment, fresh produce quality assessment, sorting and classification, and in process automation. Machine vision system developed using colour camera (monochrome or RGB colour camera) are capable of recognizing two-dimensional data attributes (pixel size) in the image such as shape, size, colour and texture in the visible colour region (Chen et al., 2002). In a recent development, machine vision equipped with hyperspectral, near infrared and infrared camera are able to inspect the internal quality of produce under the light invisible to humans such as ultraviolet (UV), near infrared (NIR) and infrared (IR). The use of machine vision techniques offers numerous potential to helps farmers to make a better management decision besides reducing the time taken to solve complex agricultural problems. As for an instance, machine vision equipped with sensing elements and machine learning techniques provides a powerful set of tools that applied for different field application in agricultural practices (Rehman et al., 2019). The systems developed are fairly accurate, nondestructive and yield consistent thus remove the possibility of human error and reduce the time taken for decision making.
Developing a machine vision in the agricultural field poses a challenging task especially in meeting the requirement of the subject for image acquisition. The task can be easy if the subject to be imaged is fairly simple and presented in a controlled environment. However, its application in solving the agricultural problems is challenging due to several factors, among others, the uncontrolled lighting in the field, overlapping and touching samples, irrelevant data in the background image and often time, the presence of higher variability in the agricultural object. Selection of the appropriate camera and lens is determined by the inspection task. Suitable resolution and sensor size, depending on the type of camera used is very important in developing a workable machine vision system. The distance of the object to the camera influences the details of the image to be captured. Homogeneous lighting system plays an important role to illuminate the object consistently for image acquisition while correct lighting selection highly influences the data captured in an image.
In the context of the rice industry in Malaysia, detection of weedy rice within the cultivated rice seed is vital due to its threat to the yield of rice during field production. Currently, rice seed testing requires laboratory workers to count, identify and distinguish between the cultivated rice seed and weedy rice seeds or off-type seeds. There is no machine vision system ever developed to address the problem of weedy rice contamination in the rice seed processing plant. Therefore, this research aimed to explore the possibility of replacing this manual process with a machine vision system to reduce the time taken for inspection and human error due to tiredness. Specifically, the emphasis of this project is to develop a laboratory-scale image recognition system that can identify weedy rice among the cultivated rice seed based on seed variety. The one-to -one model between weedy rice and cultivated rice seed variety is to be developed since the application is varietal based classification. Criterions used in developing the hardware prototype involving the image acquisition setup, lighting configuration, seed plate design and overall setup to make a workable machine vision system using an area scan camera will also be discussed in this paper.

Methodology
This research aimed to develop a machine vision prototype for weedy rice seed identification within local Malaysian cultivated rice seeds varieties using image processing and classification technique. The research gap lies in the development of the machine vision itself including the hardware setup for image acquisition, finding the right illumination module, specifying the distance between object and camera, background selection for seed holder, image processing and analysis program.

Selection of camera and optic lens
Selection of correct lenses and optics is the most important task to match the image sensor size. Required resolution of the camera can only be obtained by choosing the correct pairs of the lens. If a low-resolution lens is paired with a high-resolution image sensor, the overall machine vision system becomes limited by the resolution of the lens. As part of the image acquisition system, various size of image sensor area scan camera and lens with different specification were tested. The most important specification for the area scan camera was the resolution and pixel size because it indicates the camera ability to provide high information on a small seed image. A complementary metal oxide semiconductor (CMOS) sensor was chosen over charge couple device (CCD) sensor due to its lower cost and improved quality as near to CCD sensor.
For this research, the design constraint was established prior to the hardware selection. A maximum height of the overall system was set to not exceed 1 m. Therefore, the working distance from the front of the lens to the seed kernel on the seed plate was limited between 10 -20 cm. A sharp image is required for robust inspection and thus, narrow down the selection of the lens focal length. The longer the focal length, the narrower the angle of view, the higher the magnification. Figure 1 depicts the effect of focal length and working distance on the seed sample. It shows an image plane at a certain focal length and seed samples at a distance of H 1 , the low number of seeds image were viewed, due to higher magnification of the image size and smaller field eISSN: 2550-2166 © 2020 The Authors. Published by Rynnye Lyan Resources FULL PAPER of view (FOV). At distance H 1 , one single seed kernel consists of high pixels number. Meanwhile, at distance H 2 , the higher number of seeds image, due to bigger FOV but the lower magnification of the image size. At distance H 2 one single seed kernel consists of low pixels number. Thus, for lens selection, it is important to balance the focal length with the desired clarity at a given specified distance from the seed samples. An entocentric lens with fixed focal length 25 mm with 6 MP is paired to the 6MP CMOS area scan camera with a sensor size of 7.4 mm X 4.9 mm at 13.7 mm working distance for this project. At this distance, a shallow depth of field is maintained by sharply focusing more on the seed sample rather than the background. To allow shallow depth of field, large aperture (low F-stop) was used for the iris opening and adjusting the exposure time to 2000 µs for colour image acquisition. For the monochrome image, the black level was adjusted to 200 to influence the image contrast due to low F-stop number. Figure 2 shows the effect of clarity on different working distance with a fixed focal length using the selected area scan camera. Clarity is defined, as the image captured is able to see visible hair protrusion on the seed kernel.
To determine each of the seed kernel measurement, object size is calculated using the sensor information such as number of pixel count, pixel size working distance and the focal length of the lens as shown in Equation 1. This measurement is important in calculating the real value of morphological features of the seed kernel for image analysis and classification. Table 1 shows the specification of the selected area scan camera while Table 2 shows the choices of the lens with the specification of 6MP. The focal length of the lens varied from 8 mm to 50 mm. Different focal length requires a different working distance. The various combination between the camera and lens were tested, however, the best lens with the focal length of 25 mm was chosen to be used in this project with adequate working distance. The working distance and image clarity are at its best with the limitation on the height of the overall imaging system.

Selection of lighting
Various lighting effect as shown in Figure 3 was experimented before selecting the correct light for image acquisition. A normal LED bulb with 10 Watt at a 6500k temperature that provides a bright amount of blueish hue similar to daylight bulb was tested. However, the normal LED bulb did not provide enough brightness and uniform illumination on the FOV, thus the shadow exhibits inside the seed holes. The LED bulb was not diffused as intended to illuminate the sample area, therefore other alternatives were sought. To overcome the uneven illumination, a low angled diffused square illumination (DLW2-60-070-1W-24V) industrial LED lighting was tested as a front light and it proved to provide uniform illumination on the seed samples. Shadows were eliminated and the module used a single channel lighting controller with brightness range from 0-10. Another problem occurred that the image using only a front light will have a background of the seed holder. A high-intensity backlight (BHLX3-00-320x320-X-W-24V) was coupled together as part of the illumination    Table 3.

Seed holder design
Several factors influenced the seed plate design for this system. Seed plate size, number of holes and the dimension of the holes, background colour and the effect of the background on the image. Various designs were tested with different background and final design are as presented in Figure 4. The effect of different background colours was discussed in Ruslan et al. (2018).
The size of the seed holder was determined from the calculation of the field of view based on the maximum sensor size, focal length and working distance as depicted in Equation 2: With the selected sensor size of maximum 1/8'" (7.2 mm x 5.4 mm) and using 25 mm focal length, at a working distance of 137 mm, the field of view was 41.1 mm X 30.8 mm. Therefore, the sizing of the seed plate and number of holes in one image was designed to be limited by this FOV. Another limitation to the design of the seed holder is the selection of front light was limited to a size 10 cm x 10 cm to illuminate the seed sample. To design the seed holes, various varieties of Malaysia longgrain rice seed were tested and later decided to compromise for 13 mm long and 4 mm dimension with a circular shape to ensure the seed is non-touching and not  overlapped with each other. The background colour of the seed holder was decided to be based on clear filament colour used in the 3D printer as shown in Figure 5.

Final prototype of the machine vision
The overall design of the machine vision is easy to carry with two long handles on the side of the outer casing. All the components inside the machine vision were secured properly to the casing to ensure no unnecessary movements during transportation. The RiSe-IViS is suitable to be placed either on a bench or tabletop near to a power supply.

System integration, image processing and analysis
System integration between hardware and software of the machine vision was integrated with LabView (2016). For image acquisition and data analysis, a graphical user interface (GUI) was developed to assist end-user. In LabView, programming was carried out to acquire an image from the area scan camera. Simple image processing programme was developed for monochrome and RGB images separately. The most important steps in image processing were image segmentation. For RGB image, threshold-based segmentation was employed for each of the colour planes to separate between foreground and background. The selection of thresholding value was defined based on the RGB histogram. For example, in Red colour plane, the pixel value of 0-250 was retained and converted to binary value 1 (white) in the image segmentation process. Pixel value outside of 0-250 becomes 0 (black). For monochrome image, simple thresholding on pixel value between 0 -3500 (image format is Mono 12 thus pixel value ranges from 0 -4600) was converted into binary value 1 while values outside the region become 0. The resulted image from image segmentation was binary with 1 for the seed and 0 for the background. The pixel value corresponding to 1 was masked to display the seed image in RGB or Mono format. The image thresholding was kept consistent for all images to allow for uniform data extraction. The IMAQ particle measurement counts the particle (pixel) based on the measurement to be extracted using the masked image. The processed images were used for morphological, colour and textural features extraction. The data extracted was saved in an Excel file for future analysis in the machine learning programme. Machine learning algorithm such as neural network, discriminant analyses and other techniques were chosen and to be compared in developing the model for variety classification. Preliminary classification results based on morphology features using this prototype was reported in Ruslan et al. (2019). The RiSe-IViS is expected to classify major rice seed varieties available in Malaysia against the weedy rice variants with superior accuracy.

Conclusion
The machine vision system has great potential in solving agricultural problems. This type of system would become a very important and routine feature in the near future, given the advancement in sensing technology. The current prototype has proved that weedy rice is able to be identified among other cultivated rice seed using image classification technique. This machine vision can be improved by having a LED display panel on the front of the case once the models for the classification is developed, therefore there is no need for laptop connection. With a click of the start button on the panel, the RiSe-IViS could do the imaging and display result instantaneously. The static imaging system is also possible to be improved by converting the platform into a conveyor or chute system for faster image acquisition. Besides using area scan camera and utilizing the visible range, other sensing devices such as hyperspectral, nearinfrared, and multispectral camera are available to be explored. One of the main constraints in developing machine vision is the cost. The selection of the sensing system greatly influenced by the problem to be solved, given there are many choices of methods to be adopted. In near future, developing a machine vision will be costeffective, more accurate and robust. With the advancement in artificial intelligence, a positive future for machine vision is on progress.