BBR: An open-source standard workflow based on biophysical crop parameters for automatic Botrytis cinerea assessment in vineyards

Assessing crop diseases at an early stage allows farmers to apply chemicals at the most efficient moment to control the pest and focus on the plants with the highest risk of developing the disease. Moreover, the earlier the assessment the lower the consequences, such as yield reduction, which is crucial to feed the increasing world population. BBR is the first open-source software based on physical and phenological crop parameters that grant the early assessment of botrytis bunch rot disease, caused by Botrytis cinerea, in vineyards using UAV images. This software has been released to improve the current procedure for early botrytis bunch rot assessment, which consists of laborious and time-consuming field inspections. In addition, BBR is easily modifiable by users to implement the workflow in their fields. For future work, the software can be extended to other crops and/or other fungal diseases.


Importance of disease assessment in agriculture
There is a worldwide need to increase food production to feed the rising world's population, and agriculture is key to solving this escalating problem.Agricultural land represents 38 % of the global land surface [1], but it is a limited resource, and the global cropland area per capita is continuously decreasing.Therefore, increasing the productivity and yield of each hectare is crucial.One cause of yield reduction is the presence of crop diseases, which lower the photosynthetic accumulates produced, which finally leads to lower yield production.All diseases have a specific time window inside which the pesticide has an optimal performance [2].Thus, assessing diseases at that optimal moment, which mostly coincides with an early stage of disease development, is key to avoiding yield reduction [3].In addition, preventive treatments in the areas in which there is a high risk of developing the disease also lead to yield increase [4].
Regular field inspections are essential to assess diseases before they further develop [5].Nevertheless, scouting large areas is laborious, time-consuming, and subjective to the operator in charge.Therefore, it is important to develop tools that allow hotspot detection and evaluate the spatial variability so that farm managers, field inspections, and treatments focus on those affected or potentially affected areas.Remote sensing and more specifically unmanned aerial vehicles (UAVs) offer a great opportunity to reduce the time spent on observing the whole field and provide an objective overview of the current status of the crop considering the spatial variability within the field.
Viticulture is important in many countries in Europe due to its large contribution to the European socioeconomic sector [6].Vineyards make up 7.3 million hectares of the total area in the whole world, with almost half of that, 3.3 million hectares, being in Europe [7].
The fungus Botrytis cinerea causes botrytis bunch rot (BBR) in grapevines, which especially affects ripe berries and significantly reduces yield and wine quality [8][9][10].The presence of this disease starts in the inside of the bunches, which is not yet visible during field inspections and develops outward, being visible when it is in an advanced stage of development.

State-of-the-art software and algorithms for disease assessment
The current software and algorithms for disease assessment are mostly implementing deep learning techniques applied to multiple crops, such as tomatoes, cucumbers, and apple trees, among many others [11][12][13].As well, Mohanty et al. [14] trained a deep convolutional neural network to identify three pathogen fungi in vine plants.In addition, some studies worked with RGB and multispectral sensors to identify other fungal diseases in vineyards, such as components of the Esca complex or Flavescence dorée [15][16][17][18].However, the object detection methods implemented are trained on canopy images/videos of pathologies, when symptomatology is already present.As mentioned, Botrytis cinerea does not show leaf symptoms but grape cluster symptoms when the fungus is further developed, and not optimal for accurate pesticide application.
Furthermore, there is a lack of standardised software methods to automatise data processing workloads.The current disease assessment methods require user intervention to execute each step separately, slowing down the required time for disease risk assessment.Bock et al. [19] reviewed the current automated sensor-based methods to measure plant disease severity, concluding that visual estimation of disease severity will still be used for years alongside automated systems, with a tendency to fully automated AI-based methods for the foreseeable future.Consequently, having a whole automated process eases data analysis inside a time window that allows farmers to apply the treatment at the optimal moment.Hence, software to automatically assess botrytis at an early stage of development is crucial to avoid yield reduction in vineyards.
The assessment of fungal diseases using UAV imagery is difficult because of the leaf occlusion issue (grape clusters are often under the leaves).Moreover, when the disease is visible in the clusters, it is too late for an early assessment.Therefore, this paper presents an innovative approach: the BBR software, the first open-source standard workflow for automated BBR assessment using UAV images aiming to evaluate disease development factors [20].In pathology, a given disease is modelled Fig. 1.Workflow of the BBR software.On the left part of the diagram, the training phase is described.On the right, the testing phase is displayed.The inputs are the raw multispectral images gathered with the UAV, the Region of Interest (ROI), 2 grids (one for statistics at plant-level and the other for the shadows), and the Ground Truth (GT) collected on the field.The output of the software for the training phase is the trained Random Forest (RF) model.The outputs of the testing phase are the heatmap (in shapefile and .tiffformats), and an assessment report in PDF.Colour should be used for the printed version.using the "disease triangle", based on three components: (1) the pathogen, (2) the host, and (3) the environment, and serves to understand how the disease is established [21].Regarding the pathogen, this software assumes that Botrytis cinerea is endogenously present in the vineyard [9], so it focuses on extracting information relative to the host (CHM, NDVI, and LAI) and factors influenced by the physical environment (DTM).In this regard, the implemented software is not only based on the spectral properties of the canopy, obtained for different parts of the light spectrum but also on physical parameters, such as the terrain elevation, that boost fungal presence.In this sense, this paper implements a novel, fully automated solution for BBR disease assessment in vineyards based on the rationale proposed by Vélez et al. [22].This can assist users, mostly farm managers, to improve the current methodology to assess botrytis.Thus, they can create prescription maps based on the information generated by the software to apply the appropriate amount of pesticide, at the right time, and focus on the vine plants that require or might require the chemical, lowering the economic spending and the environmental impact.
The software is trained with the DTM, CHM, NDVI, and LAI variables using Random Forest (RF).The most important factor was LAI, followed by the CHM.However, all variables are relevant since they all have a pvalue lower than 0.05.
The remainder of this paper is organized as follows.Section 2 presents the workflow on which the software is based along with its detailed characteristics.Section 3 shows illustrative examples of the implementation of the BBR software.The validation and the impact of the presented workflow in the field of disease assessment are discussed in Sections 4 and 5, respectively.Finally, Section 6 describes the conclusions of the source code provided.

Software description
The BBR software can be executed in script mode or cross-platform inside a Python interpreter, such as PyCharm (JetBrains s.r.o., Prague, Czech Republic).It starts by generating an orthomosaic from the raw images extracted from the UAV and afterward, more products are computed to reach the two final outputs (1) a botrytis heatmap, and (2) a botrytis assessment report.
The code provided by OpenDroneMap [23] is implemented to stitch up the raw UAV images into an orthomosaic and to obtain intermediate products such as the Digital Terrain Model (DTM) and the Digital Surface Model (DSM).This code is written in bash shell language but is executed by the Python interpreter as Python language thanks to the subprocess module [24].The rest of the code is all written in Python language.
The Unmanned Aerial System used to acquire the datasets used in this study included a multi-rotor DJI Matrice 300 platform (DJI Sciences and Technologies Ltd., Shenzhen, Guangdong, China) and a Micasense RedEdge 3 sensor (AgEagle Sensor Systems Inc., Wichita, Kansas, USA).The flights were executed at 30 m height and at nadir angle, on 16 September 2021.

Software architecture 2.1.1. The model
The disease assessment model is divided into two main blocks: (1) the training phase (Fig. 1, left), and (2) the testing phase (Fig. 1, right).This experiment was carried out in a 1.06-ha commercial vineyard, Vitis vinifera cv.Loureiro, trained in vertical shoot positioning.The total size of the dataset used in this study was 153 plants, from which 91 were not infected by botrytis, and 62 were infected.The train/test split used in this study was 0.75/0.25,which represents 95 plants as training dataset and 58 plants as test dataset.All plants of the study (infected or not) were georeferenced with centimetre accuracy using a Trimble R2 Integrated GNSS system along with a TSC3 Controller (Trimble Inc., California, USA).
The main objective of the training phase is to train the Random Forest algorithm for a given dataset, i.e. a dataset acquired in another field or on another date for which ground truth (GT) data is available.RF was chosen as the supervised machine-learning classification method since 133 it has already been used effectively in several agricultural studies [25][26][27].During that stage, an orthomosaic, DSM, DTM, Canopy Height Model (CHM), Normalized Difference Vegetation Index (NDVI), and the leaf area index (LAI) extracted from the shadows, as Vélez et al. [28] proposed, are computed.The main statistics of the variables of agronomical significance (DTM, CHM, NDVI, LAI) are then obtained with the grid pattern, defined by the distance between plants and rows.Following this procedure, specific statistics at plant level are obtained.Those statistics, in the form of a CSV file (Table 1), are the input used to train the RF algorithm.The ground truth data is also used to train the Random Forest algorithm since it provides information on which plants are affected by the disease.Hence, the input of the RF algorithm is a CSV file in which every row is a different plant and every column contains the statistics information (at plant level).The column disease is the one that contains the GT information as noBOT when there is no presence of disease or Botrytis when the disease is present.

Table 1
CSV input file of the Random Forest algorithm.To train the algorithm, the column status should be present.For testing, the 'status' column should be removed since it is the variable to be predicted.Due to space limitations, some metrics of the first three variables (NDVI, DTM, and CHM) are not present, such as the minimum value, the maximum value, the mean value, and the standard deviation.The testing phase consists of applying the trained RF algorithm to a completely new dataset, understood as a dataset that has not been used to train the algorithm, to predict the status column.In this stage, the DTM, CHM, NDVI, and LAI are extracted again at plant level.The CSV file with the statistics at plant level, without the status column, is then introduced to the trained RF model to obtain a prediction of the diseased plants.The CSV file obtained is then handed to generate a heatmap, in both shapefile and .tiffformats, to analyse the hotspots of the disease.Lastly, an assessment report in PDF format is generated to provide visual information to the farm manager.

Inputs
Table 2 summarizes the required inputs to implement the BBR software.

BBR scripts
A brief description of the main function of each Python script is provided in Table 3.

Software functionalities
The BBR software is a decision support tool to provide insights to the farm manager about the health status of the vineyards regarding the botrytis disease.It has three primary focus areas: 1. Automatize the process of botrytis disease assessment in vineyards.2. Improve the field inspection workflow and facilitate a variable rate application of pesticides according to the disease heatmap.3. Ease the work of farm managers by providing a botrytis assessment report to ease their labour in assessing the disease and act consequently.The report includes the orthomosaic of the field that was flown over, some aerial images of the variables computed to calculate the risk of botrytis, and finally the normalized heatmap indicating the risk of developing the disease.
Fig. 2 shows the use case diagram of the BBR software, in which three main actors are implied: the operator, the farmer, and the UAV.These actors are responsible for planning the UAV mission and executing it.The output of this process is the acquisition of the image dataset, which is the main input of the BBR software.The next step is the actual run of the software, from which more image datasets are generated, for instance, the CHM and the NDVI maps.In the end, the final interaction with the system is the obtention of the botrytis heatmap and the assessment report.

Sample code snippets analysis
Listing 1 provides a sample code snippet of the CHM.py script.As mentioned in Table 3, the main objective of this script is to generate the CHM of the vineyard.Hence, the minimum and maximum height of the vineyard are required (Table 2).The result of this function is a .tifffile, as can be observed in Fig. 3. Listing 1. Code example to compute the Canopy Height Model of the vineyard.Code extracted from CHM.py.Colour should be used for the printed version.

Illustrative examples
To explain the main functionalities of the BBR software, the output of a middle script is reported.Moreover, to offer visual insights into how the software evolves during the whole run, a workflow with images of each script output is presented.
The output of the CHM.py script, which is a .tifffile with the Canopy Height Model of the vineyard, is shown in Fig. 3.The height values range from 0.5 to 2 m because those were the input values entered by the user.
Fig. 4 presents the workflow of the BBR software as seen by the middle products and final outputs obtained at the main steps.The orthomosaic, the DSM, and the DTM are generated from the raw images captured by the UAV.Then, those products are transformed into more middle products to train the random forest, such as the shadows, the NDVI, and the CHM.Finally, the heatmap with the probability of botrytis presence and an assessment report are generated.
The results in Table 4 show the classification matrix for the tested plants.The values indicate that 70.7 % of the plants were properly classified to the class they belong to.There was a 0.19 and 0.4 classification error for BOT and noBOT plants, respectively.
The algorithm has been validated by correlating the heatmap values of the BBR software with Ground Truth data collected on the field.There is a high correlation between both variables, (R 2 = 0.7) indicating that the software can suitably be used for estimating the risk of BBR disease in vineyards (Fig. 5) especially when this is the first code of its kind because, in the scientific literature, there is no software for BBR assessment based on UAV orthophotos.

Impact
The BBR software allows for performing optimal field inspections as it makes it possible to focus on hot spots.Moreover, it allows optimal pesticide spraying by applying the chemical at an early stage of botrytis development and only to the plants that require it or the plants with a higher risk of developing the disease.The range of models available to perform disease assessment has increased in the last few years due to the computational improvements in hardware and software.Many models apply object detection techniques to find symptomatology on the leaves or in specific parts of the plant [15][16][17][18].However, when symptomatology is visible the disease is in a late stage of development.Other methods have been studied for the early assessment of BBR disease.For instance, Liu et al. [29] analysed the implementation of a Gigahertz ultrasonic imager to detect botrytis at an early stage.Nevertheless, their method requires expensive equipment and is not feasible to be applied in large fields.Wu et al. [30] performed a study on the early detection of botrytis in eggplants using visible and near-infrared spectroscopy.They carried out experiments in a closed room and developed a method with high accuracy (around 70 %).It would be interesting to test the practicability of their method in commercial fields.Therefore, to conclude, until the moment there is no open-source workflow applicable to large commercial fields to assess Botrytis cinerea at an early stage of development using UAV imagery.The software brings significant benefits to several stakeholders, such as farm managers, researchers, and consumers, who are the final users of this model.For instance, (i) farm managers are given an automatic workflow for efficient assessment of botrytis at an early stage, being able to plan with time the pesticide application; (ii) researchers can further improve the software and extend its applications to continue progressing on optimal field inspection and precision spray application; (iii) consumers are offered a final product, in this case, grapes and wine, which have a higher quality thanks to the reduction of the amount of pesticide and fungi that the final product contains.
By now, the code is used to assess Botrytis cinerea in vineyards, but future research can include the assessment of botrytis in other crops and/or of other fungal diseases, since the four variables included in the software are generic and generated solely from UAV imagery, allowing scalability and robustness of the proposed solution to other vineyards, both red and white varieties, as long as they are trained in vertical shoot positioning.However, scalability could be a critical consideration for its wide adoption and use.Scalability concerns can be split into two categories: technical and operational.On the technical side, it's important to consider how the BBR software can handle larger vineyards and multiple sites simultaneously.As the software needs to process a large volume of high-resolution UAV imagery, robust computational resources will be required.From an operational standpoint, the deployment of UAVs over large geographic areas can present logistical complications.However, it is important to note that this is not a challenge unique to the deployment of BBR software, but is an issue present in all workflows that rely on UAV imagery and photogrammetry.Nevertheless, these challenges do not negate the potential of BBR and it promises a significant improvement over traditional methods for assessing botrytis bunch rot disease.Future iterations of this software should focus on addressing these scalability concerns to provide a universally applicable, efficient, and user-friendly tool for farmers worldwide.Nonetheless, the variables used in this study (CHM, NDVI, LAI) are important statistics that are commonly computed by farm managers to monitor the status of the vineyards, which allows easy implementation in other vineyards with different climatology to test the robustness of the software.
The experience gained in early botrytis assessment has been made available to the agronomy community by making the software opensource [20].

Conclusions
Early-stage crop disease assessment is crucial to reduce economic damage and achieve more effective disease management.This work addressed the BBR software, the first open-source fully automated standard workflow for early-stage botrytis bunch rot assessment in vineyards based on risk analysis using UAV images.The BBR software simplifies the tasks of winegrowers and field managers.At present, the procedure to determine the presence of diseases in the field involves visual inspections.These inspections are time-consuming and rely heavily on the operator's judgment.The BBR software, on the other hand, provides objective information that isn't based on operator criteria.This not only ensures consistent data but also reduces the time gap between disease assessment and chemical application.The software enables the creation of prescription maps for differential pest treatments, focusing on the plants with the highest risk of developing the disease based on probability.The dataset coupled with software, accessible online, constitutes a great opportunity for farm managers and researchers to apply the software to other vineyards affected by the fungus.Additionally, to investigate and extend the potentialities of the BBR software, some tests can be performed on other crops and/or fungal diseases in future works.

Fig. 2 .
Fig. 2. Use case diagram of the BBR software.Operator, farmer, and UAV are the main actors.After the software is run, three outputs are obtained: image datasets, a botrytis risk heatmap, and the risk assessment report.

Fig. 3 .
Fig. 3. Canopy Height Model (in m), filtered between 0.5 and 2 m (minimum and maximum height of the vine plants), obtained with the BBR software.Colour should be used for the printed version.

Fig. 4 .
Fig. 4. Input (top left), middle-products (top and bottom middle), and outputs (bottom right) of the BBR software.To generate the heatmap and the assessment report, the shadows, the NDVI, the DTM, and the CHM are used to train the Random Forest algorithm.Colour should be used for the printed version.Validation.

Table 2
Input files of the BBR software.A description of each file required is provided, along with the script that needs the input file.

Table 3
Description of the purpose of each script included in the BBR software.

Table 4
Confusion matrix obtained with the Random Forest algorithm.Testing sample: 58 plants.