LFuji-air dataset: Annotated 3D LiDAR point clouds of Fuji apple trees for fruit detection scanned under different forced air flow conditions

This article presents the LFuji-air dataset, which contains LiDAR based point clouds of 11 Fuji apples trees and the corresponding apples location ground truth. A mobile terrestrial laser scanner (MTLS) comprised of a LiDAR sensor and a real-time kinematics global navigation satellite system was used to acquire the data. The MTLS was mounted on an air-assisted sprayer used to generate different air flow conditions. A total of 8 scans per tree were performed, including scans from different LiDAR sensor positions (multi-view approach) and under different air flow conditions. These variability of the scanning conditions allows to use the LFuji-air dataset not only for training and testing new fruit detection algorithms, but also to study the usefulness of the multi-view approach and the application of forced air flow to reduce the number of fruit occlusions. The data provided in this article is related to the research article entitled “Fruit detection, yield prediction and canopy geometric characterization using LiDAR with forced air flow” [1].

This article presents the LFuji-air dataset, which contains LiDAR based point clouds of 11 Fuji apples trees and the corresponding apples location ground truth. A mobile terrestrial laser scanner (MTLS) comprised of a LiDAR sensor and a real-time kinematics global navigation satellite system was used to acquire the data. The MTLS was mounted on an air-assisted sprayer used to generate different air flow conditions. A total of 8 scans per tree were performed, including scans from different LiDAR sensor positions (multi-view approach) and under different air flow conditions. These variability of the scanning conditions allows to use the LFujiair dataset not only for training and testing new fruit detection algorithms, but also to study the usefulness of the multi-view approach and the application of forced air flow to reduce the number of fruit occlusions. The data provided in this article is related to the research article entitled "Fruit detection, yield prediction and canopy geometric characterization using LiDAR with forced air flow" [1]. © 2020 The Author(s Value of the Data First dataset for fruit detection containing annotated LiDAR based 3D data acquired from different sensor positions and under different air flow conditions. The dataset allows testing fruit detection algorithms based on LiDAR based 3D data. Precision horticulture community can benefit from these data to test methodologies with applications in yield prediction, yield mapping and canopy geometric characterization. Presented data can be used for analysing the effect of applying forced air flow and multi-view sensing for reducing the number of occlusions in fruit detection

Data repository
The repository Lfuji-air dataset (http://www.grap.udl.cat/en/publications/LFuji_air_dataset.html) includes 3D LiDAR point clouds of 11 Fuji apple trees (Malus domestica Borkh. Cv. Fuji) containing 1444 apples (Fig. 1). A total of 8 point clouds are provided for each tree, corresponding to the combinations of the following scanning conditions: The dataset includes a total of 1353 apple annotations (out of 1444 apples manually counted in field). The remaining 6.3% apples could not be identified in the point cloud because they were not visible (from a human/visual inspection). Annotations are provided in TXT format, where the first row indicates the position of the apple centre, while the following eight rows correspond to the positions of the bounding box corners.
Raw data used to generate the 3D point clouds is also provided in the dataset. This includes LiDAR data in PCAP format, and the positions of the real-time kinematics global navigation satellite system (RTK-GNSS) system in TXT format. Section 2 describes how raw data was acquired and processed to generate the described point clouds.

Code repository
The code used to process the row data and generate the georeferenced point clouds has been made publicly available at https://github.com/GRAP-UdL-AT/MTLS_point_cloud_generation. This Matlab code combines the LiDAR and RTK-GNSS raw data to obtain the 3D model of the measured trees. Section 2.3 describes the transformation matrices implemented in this code.
Additionally, the code used in Ref. [1] for fruit detection using the present dataset has also been made publicly available at https://github.com/GRAP-UdL-AT/fruit_detection_in_LiDAR_pointClouds. This code was developed to train and test the fruit detection algorithm as well as studying different sensor heights and air flow conditions to reduce the number of fruit occlusions. Both processing codes presented in this section were implemented using MATLAB® (R2018a, Math Works Inc., Natick, Massachusetts, USA).

Experimental design
Data was collected in a commercial Fuji apple orchard (Malus domestica Borkh. cv. Fuji). A total of 11 consecutive trees containing 1444 apples were scanned 3 weeks before harvesting, at 85 BBCH growth stage [2]. The experimental setup used for data acquisition was a mobile terrestrial laser scanner (MTLS) comprised of a LiDAR sensor and a RTK-GNSS (Fig. 2). Both sensors were connected to a rugged laptop used to acquire and synchronise the data by means of the acquisition time.
The LiDAR sensor was a Puck VLP-16 (Velodyne LIDAR Inc., San Jos e, CA, USA), which generates a 3D point cloud of the scanned scene in the <LiDAR> coordinate system (Fig. 2) with an accuracy of ±0.03 m (typical) at a frequency of 10 Hz (manually set). Additionally, this sensor provides the calibrated reflectance of each point (R) [3], which is a valuable information for fruit detection due to the different reflectance of apples and background [4]. The RTK-GNSS system used was a GPS1200þ (Leica Geosystems AG, Heerbrugg, Switzerland), which provides position measurements of the MTLS in <Global> world coordinates (Fig. 2) at a frequency of 20 Hz with an absolute error of 0.01/ 0.02 m (horizontal/vertical). Further specifications of the LiDAR and RTK-GNSS sensors used are detailed in Table 1. The MTLS system was mounted on an air-assisted sprayer, next to the sprayer fan, which was used to generate forced air flow and move the tree foliage. The GNSS antenna was installed at a height of 3.5 m. The LiDAR sensor was mounted vertically, with the Z <LiDAR> axis pointing to the forward direction (Fig. 2), and placed at heights of 1.8 m (H1) and 2.5 m (H2). The experimental setup was pulled by a tractor at 0.125 m s À1 forward speed and following a linear trajectory parallel to the row of trees.

Sprayer fan characterization
In order to generate forced air flow, the air-assisted sprayer operated at 18p rad s À1 (540 rpm of PTO, power take-off angular speed). At these conditions, the air flow speed at different heights and widths was characterized using an AIRMAR 200WX weather station (AIRMAR Technology Corporation, Milford, NH, USA), which measures the wind speed with an accuracy of ±0.5 m s À1 . A total of 35 measurements from a distance of 2.4 m (distance between sprayer fan and scanned trees) were performed, corresponding to the measurement of 7 height and 5 width intervals (Fig. 3). The 7 height intervals were equally distributed from 0 m to 3.5 m height, corresponding to the maximum trees  height. On the other hand, the 5 width intervals were equally distributed along 1.4 m width, which corresponds to the field-of-view of the LiDAR sensor. The speed values shown in Fig. 3 are the result of averaging 10 measurements in each position.

Point cloud generation
The LiDAR row data consist on a set of frames acquired from different positions, where each frame P < LiDAR > is a point cloud in the <LiDAR> coordinate system: x n y n z n 1 3 7 5 (1) where n denotes the number of points in the LiDAR frame.
For the generation of the 3D point cloud of all trees (Fig. 1), each frame P < LiDAR > was transformed into <Global> coordinates as follows: where the transformation matrix T < LiDAR > / < Global > can be expanded as: Because the LiDAR and the GNSS antenna were assembled in a rigid structure, the rigid transformation matrix T < LiDAR > / < GNSS > only has a translational offset: Meanwhile, the rigid transformation matrix T < GNSS > / < Global > includes a rotational, R < GNSS > , and a translational, T < GNSS > , component. As depicted in Fig. 2, the forward direction is z 0 < GNSS > . Being q and 4 the orientation angles of the vehicle around the y 0 < GNSS > (Yaw) and x 0 < GNSS > (Pitch) axes, respectively, the transformation matrix T < GNSS > / < Global > is obtained according to: Where X < GNSS > ; Y < GNSS > and Z < GNSS > denote the position of the GNSS antenna in each axis of the global coordinate system. It is worth to mention that the experimental setup did not include an inertial measurement unit (IMU); therefore, the orientation angles q y 4 were obtained by using the forward direction computed from the measurements of the RTK-GNSS receiver. Since trials were conducted in short rectilinear trajectories, the orientation of the system was assumed to be constant along the path. The resulting point clouds were manually split into a single point cloud per tree. Then each tree was manually annotated by placing 3D rectangular bounding boxes around each apple position. This process was carried out using the software CloudCompare (Cloud Compare [GPL software] v.9 Omnia) and supported by additional RGB images of the tested trees.