Raspberry Pi Powered Imaging for Plant Phenotyping

Premise of the study: Image-based phenomics is a powerful approach to capture and quantify plant diversity. However, commercial platforms that make consistent image acquisition easy are often cost-prohibitive. To make high-throughput phenotyping methods more accessible, low-cost microcomputers and cameras can be used to acquire plant image data. Methods and Results: We used low-cost Raspberry Pi computers and cameras to manage and capture plant image data. Detailed here are three different applications of Raspberry Pi controlled imaging platforms for seed and shoot imaging. Images obtained from each platform were suitable for extracting quantifiable plant traits (shape, area, height, color) en masse using open-source image processing software such as PlantCV. Conclusion: This protocol describes three low-cost platforms for image acquisition that are useful for quantifying plant diversity. When coupled with open-source image processing tools, these imaging platforms provide viable low-cost solutions for incorporating high-throughput phenomics into a wide range of research programs.


INTRODUCTION
Image-based high-throughput phenotyping has been heralded as a solution for measuring diverse

Raspberry Pi Initialization: This work describes three protocols (Appendices 2-4) that utilize
Raspberry Pi computers for low-cost image-based phenotyping and gives examples of the data they produce. Raspberry Pi computers can be reconfigured for different phenotyping projects and can be easily purchased from online retailers. The first application is time-lapse plant imaging (Appendix 2); the second protocol describes setup and use of an adjustable camera stand for top-view photography (Appendix 3); and the third project describes construction and use of an octagonal box for acquiring plant images from several angles simultaneously (Appendix 4). For all three phenotyping protocols, the same protocol to initialize Raspberry Pi computers is used and is provided in Appendix 1. The initialization protocol in Appendix 1 parallels the Raspberry Pi Foundation's online documentation and provides additional information on setting up passwordless secure shell (SSH) login to a remote host for data transfer and/or to control multiple Raspberry Pis. Passwordless SSH allows one to pull data from the data collection computer to a remote server without having to manually enter login information each time.
Reliable data transfer is an important consideration in plant phenotyping projects because, while it is possible to process image data directly on a Raspberry Pi computer, most users will prefer to process large image datasets on a bioinformatics cluster. Remote data transfer is especially important for time-lapse imaging setups, such as the configuration described in Appendix 2, because data can be generated at high frequency over the course of long experiments, and thus can easily exceed available disk space on the micro secure digital (SD) cards that serve as local hard-drives. Once one Raspberry Pi has been properly configured and tested, the fully configured operating system can be backed up, yielding a disk image that can be copied ("cloned") onto as many additional SD cards as are needed for a given phenotyping project (Appendix 1). Time-lapse Imaging: Time-lapse imaging is a valuable tool for documenting plant development and can reveal differences that would not be apparent from endpoint analysis.

Raspberry Pi
Raspberry Pi computers and camera modules work effectively as phenotyping systems in controlled-environment growth chambers; and low cost of Raspberry Pi computers allows this approach to scale well. Growth chambers differ from (agro)ecological settings but are an essential tool for precise control and reproducible experimentation (Poorter et al., 2016) .
Time-lapse imaging with multiple cameras allows for simultaneous imaging of many plants and can capture higher temporal resolution than conveyor belt and mobile-camera systems. Appendix 2 provides an example protocol for setting up the hardware and software necessary to capture plant images in a growth chamber. The main top-view imaging setup described is aimed at imaging flats or pots of plants in a growth chamber. We include instructions for adjusting the camera-plant focal distance (yielding higher plant spatial resolution) and describe how to adjust the temporal resolution of imaging. The focal distance can be optimized to the target plant, trait, and degree of precision required; large plant-camera distances allow a larger field of view, at the cost of lower resolution. For traits like plant area, where segmentation of individual plant organs is not critical, adjusting the focal length might not be necessary. Projected leaf area in top-down photos correlates well with fresh and dry weight, especially for relatively flat plants such as Arabidopsis thaliana (Leister et al., 1999) . A stable and level imaging configuration is important for consistent imaging across long experiments and to compare data from multiple Raspberry Pi/Camera ris. Although there is more than one way to suspend Raspberry Pi/Camera rigs in a flat and stable top-view configuration, AC power socket adapters were attached to the the back of cases with silicone adhesive (Appendix 2). Raspberry Pi boards and cameras were then encased and screwed into the incandescent bulb sockets built into the growth chamber ( Figure 1).
Users with access to a 3D printer may prefer to print cases, so we have provided a link to instructions for printing a suitable case (with adjustable ball-joint Raspberry Pi camera module mount) in Appendix 2. This type of 3D printed case also works well for side-view imaging of plants grown on plates (Huang et al., 2016;Mutka et al., 2016) . For this top-down imaging example, twelve Raspberry Pi/Camera rigs were powered through two USB power supplies drawing power (via extension cord and surge protector) from an auxiliary power outlet built into the growth chamber. Time-lapse imaging was scheduled at five-minute intervals using the software utility cron. A predictable file naming scheme that includes image metadata (field of view number, timestamp, and a common identifier) was employed to confirm that all photo timepoints were captured and transferred as scheduled. Images were pulled from each Raspberry Pi to a remote server twice per hour (using a standard utility called rsync) by a server-side cron process using the configuration files described in Appendix 2.
Optimizing imaging conditions for maximum consistency can simplify downstream image processing. To aid in image normalization during processing, color standards and size markers can be included in images. Placing rubberized blue mesh (Con-Tact Brand, Pomona, California, USA) around the base of plants can sometimes simplify segmentation (i.e. distinguishing plant foreground pixels from soil background pixels), though this was not necessary for the A. thaliana example described here. Care should be taken to ensure that large changes in the scene (including gradual occlusion of blue mesh by leaves) do not dramatically alter automatic exposure and color balance settings over the course of an experiment. If automatic exposure becomes an issue, camera settings can be manually set (see Appendix 4). In this example, cameras and flats were set up to yield a similar vantage point (a 4 x 5 grid of pots) in each field of view, such that very similar computational pipelines can be used to process images from all twelve cameras. An example image has been processed with PlantCV (Fahlgren et al., 2015) , and the analysis script is available at https://github.com/danforthcenter/apps-phenotyping ( Figure 2).
Raspberry Pi Camera Stand: An adjustable camera stand is a versatile piece of laboratory equipment for consistent imaging. Appendix 3 is a protocol for pairing a low-cost home-built camera stand with a Raspberry Pi computer for data capture and management. The camera stand (79 cm width x 82.5 cm height) was built from aluminum framing (80/20, Columbia City, Indiana, USA) to hold a Nikon Coolpix L830 camera via a standard mount (Figure 1). For this application, we prefer to use a single-lens reflex (SLR) digital camera (rather than a Raspberry Pi camera module) for adjustable focus and to improve resolution. The camera was affixed to a movable bar, so the distance between camera and object can be adjusted up to 63 cm. A Python script that utilizes gphoto2 (Figuière and Niedermann, 2017) for data capture and rsync for data transfer to a remote host is included in the protocol (Appendix 3). When the 'camerastand.py' script is run, the user is prompted to enter the filename for the image. The script verifies that the camera is connected to the Raspberry Pi, acquires the image with the SLR camera, retrieves the image from the camera, renames the image file to the user-provided filename, saves a copy in a local Raspberry Pi directory, and transfers this copy to the desired directory on a remote host. As image filenames are commonly used as the primary identifier for downstream image processing, it is advised to use a filename that identifies the species, accession, treatment, and replicate, as appropriate . The Python script provided appends a timestamp to the filename automatically. We regu larly use this Raspberry Pi Camera Stand to image seeds, plant organs (e.g. inflorescences), and short-statured plants. For seed images, a white background with a demarcated black rectangular area ensures that separated seeds are in frame, which speeds up the imaging process.
Color cards (white, black, and gray; DGK Color Tools, New York, New York, USA) and a size marker to normalize area are also included in images to aid in downstream processing and analysis steps. It is advised to use the same background, and, if possible, the same distance between object and camera for all images in an experimental set. However, including a size marker in images can be used to normalize data extracted from images if the vantage point does change. Chenopodium quinoa (quinoa) seed images are shown as example data from the camera stand ( Figure 2). Seed images acquired with the camera stand were processed using PlantCV (Fahlgren et al., 2015) to quantify individual seed size, shape, color, and count; these types of measurements are valuable for quantifying variation within a population. This overall process (Appendix 3) provides a considerable cost savings relative to paying for seed imaging services or buying a commercial seed imaging station. To facilitate data acquisition and transfer on all four Raspberry Pis, scripts are written so the user only needs to interact with a single 'master' Raspberry Pi (here the master Raspberry Pi is named 'octagon'). From a laptop computer one would connect to the 'master' pi via SSH, then run the 'sshScript.sh' on that Pi. The 'sshScript.sh' script triggers the image capture and data transfer sequence in all four Raspberry Pis and appends the date to a user inputted barcode . When the 'sshScript.sh' script is run, a prompt asks the user for a barcode sequence. The barcode can be inputted manually, or, if a barcode scanner is available (we use a Socket 7Qi barcode scanner), a barcode scanner can be used to input the filename information. Again, it is advised to use a plant barcode that identifies the species, accession, treatment, and replicate, as appropriate. Once a barcode name has been inputted, another prompt asks if the user would like to continue with image capture. This pause in the 'sshScript.sh' script gives the user the opportunity to place the Low-cost computers and components are especially important since some experiments might test harsh environmental conditions and need to be replaced long-term. Each of the platforms were built and programmed in large part by high-school students, undergraduates, or graduate students and do not require a large investment of time to build or set-up. Automation increases the consistency of image and metadata capture, which streamlines image segmentation ( Figure 2) and is thus preferable to manual image capture. Furthermore, the low cost of each system and the flexibility to reconfigure Raspberry Pi computers for multiple purposes makes automated plant phenotyping accessible to most researchers.

CONCLUSION
The low-cost imaging platforms presented here provide an opportunity for labs to introduce phenotyping equipment into their research toolkit, and thus increase the efficiency, reproducibility, and thoroughness of their measurements. These protocols make high-throughput phenotyping accessible to researchers unable to make a large investment in commercial

APPENDICES
Appendix 1. Initializing a Raspberry Pi for phenotyping projects. The camera stand, growth-chamber imaging stations, and multi-image octagon phenotyping platforms that are described in detail in Appendices 2-4, use Raspberry Pis to trigger image acquisition, append metadata to filenames, and to move data. The following are the required parts and steps to initialize a single Raspberry Pi. The initialization protocol is based on the installation guidelines from the Raspberry Pi Foundation, which are under a Creative Commons license (https://www.raspberrypi.org/documentation/).

Raspberry Pi single-board microcomputer
Micro USB power supply Mini Secure Digital (SD) card, we recommend 16GB HDMI Monitor, HDMI cable, keyboard, and mouse (for initialization process)

General Raspberry Pi Initialization Protocol:
1. Install 'Raspbian Stretch with Desktop' (here version 4.9 is used, but the latest version is recommended) onto the SD card by following the installation guide at https://www.raspberrypi.org/downloads/raspbian/ 2. Insert mini SD card into Raspberry Pi and plug in monitor, keyboard, and mouse to Raspberry Pi. 3. Plug in Micro USB power supply and connect to power. The Raspberry Pi will boot to the desktop interface, which is also known as the graphical user interface (GUI Alternatively, use an Ethernet cable connection. 6. Optionally, make a local copy of the scripts that accompany this paper. In Terminal, change directory to the Desktop by typing "cd Desktop". Then type "git clone https://github.com/danforthcenter/apps-phenotyping.git" . If prompted with "The authenticity of host 'remote-host' can't be established (...) Are you sure you want to continue connecting?" enter "yes". This will download the project scripts and examples for all three phenotyping platforms (Appendices 2-4). Some of these scripts may need be adjusted after they have been cloned on the Raspberry Pi, as described below.

Raspberry Pi SD card cloning protocol:
Once you have gone through the initialization protocol for one Raspberry Pi the disk image of the SD card from that Raspberry Pi can be cloned if you need additional Raspberry Pis for your project. Any project specific scripts that need further adjustments on individual Raspberry Pis can then be completed (see Appendices 2 to 4). Cloning an SD card will generate a file of the exact size of the SD card (e.g. 16 GB). Make sure the SD cards to which the original image will be cloned are at least the same size as the original initialized card (e.g. 16 GB or bigger).

To clone an SD card on a Windows computer:
1. Download and install Win32 Disk Imager from https://sourceforge.net/projects/win32diskimager/ 2. Before opening the Win32 Disk Imager, insert the SD card (in an SD card reader if needed) from the initialized Raspberry Pi into your computer. 3. Open Win32 Disk Imager. 4. Click on the blue folder icon. A file explorer window will appear. 5. Select the directory to store the SD card image, and provide a filename for the image. 6. Click Open to confirm your selection. The file explorer window will close. 7. Under Device, select the appropriate drive letter for the SD card. 8. Click the Read button. 9. Once the image is created, a 'Read Successful' message will appear. Click OK. 10. Eject the SD card, and close Win32 Disk Imager. 11. Insert the new SD card where the image will be cloned. Make sure this SD card has as much or more storage capacity as the SD card from the initialized Raspberry Pi that was imaged. 12. Reopen Win32 Disk Imager. 13. Click on the blue folder icon, and select the image that was just created. 14. Under Device, select the appropriate drive letter for the SD card where the image will be cloned. 15. Click the Write button. Click OK. 9. Eject the SD card. 10. Insert the new SD card where the image will be cloned. Make sure this SD card has as much or more storage capacity as the SD card from the initialized Raspberry Pi that was imaged. 11. Under Pi-Crust: Select SD-Card or USB drive, select the SD card where the image will be cloned. 12. Click on Restore Backup. 13. Browse and select the image that was just created. 14. Click OK. 15. Once the SD card is cloned, a 'Your ApplePi is ready!' message will appear. Click OK. 16. Eject the SD card, and insert it into the Raspberry Pi. The Raspberry Pi is now initialized.

To clone an SD card on a Linux computer:
This protocol is adapted from The PiHut (https://thepihut.com/blogs/raspberry-pi-tutorials/17789160-backing-up-and-restoring-yo ur-raspberry-pis-sd-card) and Make sure to remove the partition name to image the entire SD card (e.g. use /dev/sdb instead of /dev/sdb1). 5. There is no progress indicator, so wait until the command prompt reappears. 6. Unmount the SD card by typing: sudo umount /dev/SDCardName. 7. Remove the SD card. 8. Insert the new SD card where the image will be cloned. Make sure this SD card has as much or more storage capacity as the SD card from the initialized Raspberry Pi that was imaged. 9. Use the command 'df -h' again to discover the new SD card name, or names if there is more than 1 partition. 10. Unmount every partition using the command 'sudo umount /dev/SDCardName' (e.g. sudo umount /dev/sdb1). 11. Copy the initialized Raspberry Pi SD card image using the command 'sudo dd if=/path/to/SDCardImage.img of=/dev/SDCardName'. 12. There is no progress indicator, so wait until the command prompt reappears. 13. Unmount the SD card, and insert it into the Raspberry Pi. The Raspberry Pi is now initialized.

General instructions for using a SSH keys for passwordless connection to a remote host:
These instructions are to allow a Raspberry Pi computer to access a remote host (e.g. another Raspberry Pi computer, bioinformatics cluster, or other computer), without having to enter login information. If a user would like to pull data from Raspberry Pi to a remote host, rather than pushing data from a Raspberry Pi to a remote host, similar instructions would be followed on the remote computer. 1. In the Terminal window, enter "ssh-keygen" to create a public SSH key for passwordless access to a remote host. 2. Press enter to use the default location when asked to "Enter file in which to save the key". 3. Press enter two times, to use no passphrase. 4. Optionally, enter "ls /.ssh" to verify the SSH key was generated. The files "id_rsa" and "id_rsa.pub" should be listed.

5.
Use the command "ssh-copy-id -i /.ssh/id_rsa.pub user@remote-host" in the Terminal window to copy the public SSH key to the remote host, where "user@remote-host" should be replaced by the name of the remote host where the images will be stored (e.g. ssh-copy-id -i /.ssh/id_rsa.pub jdoe@serverx). 6. If prompted with "The authenticity of host 'remote-host' can't be established (...) Are you sure you want to continue connecting?" enter "yes". 7. Enter the user's password for the remote server, if prompted. 8. Verify the SSH key was successfully copied to the remote host. SSH to the remote host from the Raspberry Pi, using the command "ssh user@remote-host" in the Terminal window (e.g. ssh jdoe@serverx). No password should be required if the key was copied successfully.

Additional Notes:
• For official distributions of the Raspbian Raspberry Pi operating system, the default username is "pi", and default password is "raspberry". The following protocols assume these default settings are unchanged.

Mounting of Raspberry Pi/Camera rigs for top-view time-lapse imaging:
1. Connect a camera module to each Raspberry Pi, as described in Appendix 1.
2. Attach the AC light bulb socket adapters to the Raspberry Pi cases with silicone adhesive, and install an initialized Raspberry Pi computer (see Appendix 1) and camera in each case. 3. Plug each of the Raspberry Pis into the monitor, keyboard, mouse, and USB power. Following Appendix 1, change the hostnames of the Raspberry Pis. For example, the sample configuration files we provide assume that the twelve Raspberry Pi have hostnames 'timepi01', 'timepi02, ... 'timepi12'. Recall that protocol 1 describes how to copy these configuration files, e.g. to /home/pi/Desktop. 4. Make a folder for images on each of the Raspberry Pis. In our example we made a folder '/home/pi/images'. To do this on the Terminal type: 'mkdir /home/pi/images'. If you wish to use a different path image folder, change line 14 in the example file 'pull-images-from-raspi.crontab' (within the appendix.2.time-lapse subdirectory). You will also need to change the 'photograph-all-5min.crontab' and 'photograph-all-5min-vhflipped.crontab' lines 12, 15, 18 ,21, and 24 so that the images are saved to the correct location during acquisition (both of these scripts are located in Desktop/apps-phenotyping/ appendix.2.time-lapse ) 5. Physically position each Pi/Camera rig within an experimental growth space (e.g. by screwing adapters into sockets or joining velcro strips together). Take photos (with the raspistill command) to confirm that each camera covers a suitable field of view. See additional notes below on taking photos remotely and optionally flipping photo orientation.

Starting and ending a single imaging experiment:
6. The 'photograph-all-5min.crontab' and 'photograph-all-5min-vhflipped.crontab' files are cron tables and contain the commands that trigger regular image capture. Both scripts are currently written to capture data every 5 minutes between the hours of 8:30 and 17:30 (8:30am to 5:30pm standard time). If that frequency is too high, the first number or comma separated list of numbers on lines 12, 15, 18, 21, and 24 have to be altered to reflect that change. If the hours of imaging are different, then the second number or range of numbers on lines 12, 15, 18, 21, and 24 has to be altered. 7. The 'photograph-all-5min.crontab' and 'photograph-all-5min-vhflipped.crontab' files also control wifi, turning off outside of the imaging window/photoperiod (see comments in files). Wifi is set to turn on ten minutes before the start of imaging and turn off 10 minutes after imaging ends. If the minute or hour of imaging is different from our experimental setup then the first two numbers on both lines 37 and 43 have to be altered. 8. Once both 'photograph-all-5min.crontab' and 'photograph-all-5min-vhflipped.crontab' scripts are satisfactory, install the cron jobs on each Raspberry Pi. The 'install-twelve-crontabs.sh' script (run from a remote machine, and depending on reasonable wifi connectivity) does this for all twelve Raspberry Pis, but first the user has to determine if the images need to be flipped or not. The difference between the 'photograph-all-5min.crontab' and 'photograph-all-5min-vhflipped.crontab' scripts is that the 'photograph-all-5min-vhflipped.crontab' imaging command flips the images in both the vertical and horizontal directions. Flipping the images might be necessary if there are differences in the orientation of the cameras, and thus images, between the Raspberry Pis. If a Raspberry Pi's images are in 'wrong' orientation, open the 'install-twelve-crontabs.sh' file and follow the directions for commenting and uncommenting. 9. To pull data from the Raspberry Pi computers to a remote host, line 15 of the 'pull-images-from-raspi.crontab' must be changed to the path that you would like the images to go to on the remote host. The remote host must have passwordless SSH set up so that it can login to the Raspberry Pis without a password. This is very much like the 'general instructions to generate a SSH key for passwordless SSH to a remote host' in Appendix 1, but in reverse. Briefly, on the remote host you would generate an ssh key (command "ssh-keygen" ), then copy that to the Raspberry Pi (e.g. "ssh-copy-id -i /.ssh/id_rsa.pub pi@timepi01"). 10. Once the RASPIDIR and SERVERDIR paths are changed in 'pull-images-from-raspi.crontab', put the ''pull-images-from-raspi.crontab' file on the remote host, then install it on the remote host on the command line by typing: 'crontab pull-images-from-raspi.crontab'. Warning: this will overwrite existing cron jobs. 11. Upon conclusion of an experiment, suspend photography on each Raspberry Pi by "removing" the active crontab (crontab -r). Once the experiment is done, you can safely shutdown the Raspberry Pis using the 'shut-down-all.sh' script, if desired. 12. If a cron table job is set up on the Raspberry Pi it will take images as long as the Raspberry Pi has power, disk space, and a functioning camera module.