Automated grapevine cultivar classification based on machine learning using leaf morpho-colorimetry, fractal dimension and near-infrared spectroscopy parameters

Highlights

• Grapevine leaves were used to develop machine learning models.

• A cultivar classification model based on morpho-colorimetry was highly accurate.

• A second cultivar classification model based on NIR spectroscopy had similar accuracy.

• Comparisons between these two models and techniques were provided.

Abstract

The application of computer vision algorithms and chemometric fingerprinting using near-infrared spectrometry (NIR) of plant leaves, offers enhanced capabilities for ampelography by providing more accurate methods to discriminate leaves based on morphological parameters, and chemometrics, respectively. This paper showed that machine learning algorithms based on morpho-colorimetric parameters and NIR analysis separately, were able to automatically classify leaves of 16 grapevine cultivars. The artificial neural network (ANN) model developed with morpho-colorimetric parameters as inputs (Model 1), and 16 cultivars as targets, rendered an accuracy of 94% to classify leaves for all cultivars studied. The ANN model obtained with the NIR spectra per leaf as inputs (Model 2), and the real classification as targets, rendered 92% accuracy. The automatic extraction of morpho-colorimetric data, NIR chemical fingerprinting and machine learning modelling rendered rapid, accurate and non-destructive methods for cultivar classification, which can aid management practices.

Introduction

Grapevine leaves of different cultivars vary in chemical composition and morphology such as shape, dimension, color and edge shape. These differences in morphometric characteristics have been acquired as evolutionary traits corresponding to specific gene expressions and their interaction with the environment to which each cultivar has been adapted to (Nicotra et al., 2011, Vlad et al., 2014). Therefore, every leaf morphology and chemical parameter is unique for all cultivars, which allows their classification through a series of different observations or measurements for identification purposes (Chitwood et al., 2014). Besides the typical morphometric measurements such as area, petiole size, perimeter, eccentricity and edge shape, the fractal dimension has been considered as a robust classification parameter. In a study on Sangiovese grapevine genotypes (2001b), the fractal dimension and the box-counting method provided a more objective approach in the classification of these genotypes as it can easily capture the complexity of the leaf structure. This remedy the misclassifications commonly occurring in the conventional methods due to the existence of heterogeneous leaves, homonyms and synonyms.

Ampelography is the characterization and classification of grapevine cultivars using either color or shape of leaves, or their fruits or berries, with early origins in France (Rendu, 1857). The first systematic attempt of classification using ampelography methods was proposed after the second world war (Galet, 1968). In the early days of this technique, experts, whom were familiar with all the morphological characteristics of leaves and berries from different cultivars, were able to identify and name them; however, this method is considered very subjective and inaccurate (Backhaus et al., 2010). Nowadays, there have been a few improvements in the classification of grapevine cultivars using more objective methods, mainly through chemical, spectroradiometry, and genetic fingerprinting techniques (García-Muñoz et al., 2012). However, these techniques require high level of specialized skills, specific and expensive instrumentation, and tedious laboratory work, which renders their generalized practical use difficult and inaccessible for routine grapevine cultivar classification. With the advancement in digital photography and image analysis algorithms, it is possible to obtain semi-automatically a series of morphological parameters that have been already applied for grapevine leaves (Alessandri et al., 1996, Bodor et al., 2013, Fourie, 2012) and from Arabidopsis leaves (Backhaus et al., 2010). Nevertheless, these semi- and automatic methods have not yet applied color parameters in their algorithms as part of their output variables as other authors such as Orru et al. (2012) have included for grapevine seeds classification. Furthermore, none of these techniques offered an automated classification, rather they are only descriptive, showing different cultivars or clones clustered and separated using multivariate statistical methods such as principal component analysis (PCA) and cluster analysis.

Feature extraction from objects using image analysis algorithms or morphometrics, have been widely used for different plant species based on different parameters used for morphometric analysis (Backhaus et al., 2010) such as leaf shape, elliptic Fourier descriptors, contour signatures, landmark and linear signatures, shape features, polygon fitting and fractal dimensions, venation extraction and analysis, leaf margin analysis and leaf texture analysis (Cope et al., 2012). Furthermore, color extraction from leaves has been used for different applications such as disease identification in grapevine leaves (Meunkaewjinda et al., 2008). Studies based on leaf features and color extraction (morpho-colorimetrics) can be descriptive through multivariate data analysis assessing the main features that separate different leaves either by shape, contour or color descriptors. However, there is an increasing interest in the separation of leaves automatically using machine learning algorithms either for identification, classification, or for detection of biotic and abiotic stresses (Backes et al., 2012, Bruno et al., 2008, Meunkaewjinda et al., 2008, Pandolfi et al., 2009a, Rossatto et al., 2011, Wu et al., 2007).

Leaves can also be analyzed using reflectance of light spectra from the leaf surface and detected using near-infrared spectroscopy (NIR). The NIR technique has been used as a chemical fingerprint to characterize leaves, and to obtain proxy data that can be robustly correlated to different compounds and nutrients (De Bei et al., 2017, Susan et al., 1998), and water status (De Bei et al., 2011, Santos and Kaye, 2009).

This paper describes the automated extraction of morpho-colorimetric and fractal dimension (FD) features from scanned mature leaves of 16 different grapevine cultivars by using image analysis and computer vision algorithms through a customized code written in Matlab® ver. R2017b (Mathworks Inc., Natick, MA. USA). NIR was also used to obtain the chemical fingerprinting of the leaves to compare the accuracy of classification based on both the morpho-colorimetric data and the chemical fingerprinting. Principal component analysis (PCA) was executed to visualize the classification of the leaves based on the parameters measured. Finally, pattern recognition models using artificial neural networks (ANN) were developed to automatically classify each cultivar using morpho-colorimetry and chemical fingerprinting.