Next Article in Journal
Landslide Susceptibility Mapping: Machine and Ensemble Learning Based on Remote Sensing Big Data
Next Article in Special Issue
Long-Term Grass Biomass Estimation of Pastures from Satellite Data
Previous Article in Journal
Landslide Detection with Himawari-8 Geostationary Satellite Data: A Case Study of a Torrential Rain Event in Kyushu, Japan
Previous Article in Special Issue
Analysis of the Recent Agricultural Situation of Dakhla Oasis, Egypt, Using Meteorological and Satellite Data
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Assessing the Accuracy of Multiple Classification Algorithms for Crop Classification Using Landsat-8 and Sentinel-2 Data

1
Institute of Regional Development, University of Castilla-La Mancha, 02071 Albacete, Spain
2
Higher Polytechnic School of Ávila, University of Salamanca, Av. de los Hornos Caleros, 50, 05003 Ávila, Spain
*
Author to whom correspondence should be addressed.
Remote Sens. 2020, 12(11), 1735; https://doi.org/10.3390/rs12111735
Submission received: 31 March 2020 / Revised: 22 May 2020 / Accepted: 25 May 2020 / Published: 28 May 2020

Abstract

:
The launch of Sentinel-2A and B satellites has boosted the development of many applications that could benefit from the fine resolution of the supplied information, both in time and in space. Crop classification is a necessary task for efficient land management. We evaluated the benefits of combining Landsat-8 and Sentinel-2A information for irrigated crop classification. We also assessed the robustness and efficiency of 22 nonparametric classification algorithms for classifying irrigated crops in a semiarid region in the southeast of Spain. A parcel-based approach was proposed calculating the mean normalized difference vegetation index (NDVI) of each plot and the standard deviation to generate a calibration-testing set of data. More than 2000 visited plots for 12 different crops along the study site were utilized as ground truth. Ensemble classifiers were the most robust algorithms but not the most efficient because of their low prediction rate. Nearest neighbor methods and support vector machines have the best balance between robustness and efficiency as methods for classification. Although the F1 score is close to 90%, some misclassifications were found for spring crops (e.g., barley, wheat and peas). However, crops with quite similar cycles could be differentiated, such as purple garlic and white garlic, showing the powerfulness of the developed tool.

Graphical Abstract

1. Introduction

The use of remotely sensed data to perform crop classification is a complex task. However, it is a useful tool for planning and management of many agriculture’s activities like irrigation, among many others [1]. Moreover, the precise and time appropriate information on crop type and surface provided by the remote sensing data are considered the key to forecast the crop production [2]. Therefore, crop classification is the basis for many environmental and socioeconomic applications [3]. Several studies have applied machine learning techniques to classify crop types, for instance support vector machines [4,5,6], neural network [6,7,8], maximum likelihood [9], random forest [4,7] or decision trees [6].
Some applications of crop classification using Earth observation (EO) techniques are: (1) monitoring irrigated land, primarily in semiarid and arid regions [10]; (2) crop monitoring for proper funds allocation, such as subsidies for the Common Agriculture Policy [11]; (3) detection of seasonal crop abandonment [12] and (4) changes in cropping patterns along the years [13].Further, there are many other applications primarily focused on water and land management and governance.
Different sources of information are currently available from both open access and private sources, which empower the capabilities of EO techniques for crop classification tasks. The launch of Sentinel-2A and B satellites has boosted the development of many applications that could benefit from the fine resolution of the supplied information. However, it is still necessary to analyze the contribution of this fine information in vegetation classification [14]. Additionally, the interoperability between Sentinel-2A (MSI) and Landsat-8 (OLI), as the most commonly used remote sensing information sources, must be evaluated [15]. Some efforts have been made in the evaluation of the Sentinel-2A products capabilities for vegetation classification [14]. This preliminary analysis concluded that the main bands that contribute to proper crop classification are located in the red edge and the ones with fewer contributions in the near infrared. The series of spectral bands of Sentinel-2 in the red-edge and shortwave infrared (SWIR; Band 5, Band 6, Band 7, Band 10, Band 11 and Band 12) are exceptionally valuable particularly for the investigation of agriculture and vegetation mapping [14]. It is easy to provide a series of indexes based on the use of these different bands, which are related to the presence of chlorophyll, useful for the discrimination and classification of the type of covering present on a territory [16]. Given that the temporal orbit cycle of Sentinel-2 was especially designed to perform combined observation with Landsat-8 [17], an increase in the crop monitoring mission became possible. Consequently, one of the objective of this manuscript is to evaluate the use the multi-temporal Sentinel-2A and Landsat-8 data to take advantage from their interoperability [15,18] for crop classification.
Accessible and open remote sensing information has led to applications dealing with multi-temporal data, which restricts the results to the end of the crop season. To analyze the effect of the number of dates with the aim of early crop classification during the crop season, [19] evaluated a number of different images along the crop season to determine the earliest crop classification and the improvement in the overall accuracy for the different dates. These authors concluded that using the available information up to mid-July could result in high accuracy (86%) in crop classification, reaching 92% at the end of the season. Additionally, they used the enhanced vegetation index EVI instead of the normalized difference vegetation index NDVI to avoid environmental distortion.
Vegetation indices (VIs) derived from reflectance data acquired from optical sensors have been used over a wide range of scenarios to assess variations in the physiological states and biophysical properties of vegetation [20,21], particularly, the normalized difference vegetation index NDVI [22], which is one of the most used indices in remote sensing studies [23]. NDVI measurements incorporate observations of different plants and can be used to present descriptive characteristics of phenological stages [24].
A great effort has been made in the selection of proper algorithms for crop classification, from the use of traditional parametric algorithms, such as maximum likelihood, to nonparametric algorithms, such as neural networks, support vector machines, decision trees and random forest among many others [3]. Table 1 summarizes some experiences in crop classification with different classification algorithms, sensors, number of classified crops and locations, which complement those referenced in [3].
This study evaluates the contribution of Sentinel-2A information to crop classification over the use of Landsat-8 images in a semiarid region located in the Southeast of Spain. For the same area and data, a classification by using only Landsat-8 images, only Sentinel-2A and both sources of information was evaluated. We also compared 22 classification algorithms, all of them nonparametric, to determine which one is the most robust and efficient. Additionally, we analyzed the interoperability between Sentinel-2A and Landsat-8 and its influence on the classification results.

2. Materials and Methods

2.1. Overview of the Methodology

To assess the potential of Landsat-8 and Sentinel-2A information for the improvement of crop classification we followed the classical main steps (Figure 1) for crop classification tasks described by [3]:
Data collection. This step covers the selection of remotely sensed and ground truth data. In this study, we selected Landsat-8 and Sentinel-2A images that cover the case study area during 2016. Concerning the data preprocessing, we did not perform any atmospheric correction as we used reflectance at the top of the atmosphere (ToA) to calculate VIs. Crop classification is based on the temporal pattern of VIs and not on its absolute value. Data collection step also comprises collecting the ground truth data from the field visits, which greatly influences the accuracy of the classification procedure.
Data preparation, which consistsofgenerating VIs, combining VIs from different sources of information (Sentinel-2A and Landsat-8, in this case) and performing statistics of the VIs for each ground truth plot. In this case, selected VI was NDVI, because it is the most widespread used VI and very accurate in the monitoring of the crop phenology. To combine NDVI values from different sources (Sentinel-2A and Landsat-8) a comparison of the mean values of NDVI [32] for each plot and each source of information (after removing border effect and other artifacts) was made. Then, Landsat-8 NDVI values were corrected using the obtained linear relationship between both sources of information. To train the classification algorithm, a wide database of ground truth data was acquired, being one of the main strengths of this study. To ensure high quality input data, ground truth was selected to consider plots with more than 1 ha, eliminate border effect (using a buffer of 30 m) and calculating the mean and standard deviation of the VI for each plot. These values represent the pattern of the vegetation in the plot and, therefore, return the information to perform crop classification.
Classification process, which was carried out using field visits during 2016 of 2032 plots for 12 crops. We calibrated and evaluated the performance of 22 nonparametric algorithms.
Quality assessment and selection of the best classification algorithm. With the 30% of the ground truth data, a quality assessment was done based in the generation of the confusion matrix and classical performance indicators of the overall accuracy, producer’s accuracy, user’s accuracy and the F1 score [33]. The evaluation of selection of the best classification algorithm was performed based on these performance indicators.

2.2. The Case Study

The proposed methodology was carried out in the hydrological unit (H.U.) 08.129 (Figure 2) with a total surface area of 7200 km2. This extension can be considered a wide area that can provide trustful results compared with other studies performed in smaller areas [34,35,36,37,38]. The H.U. is located in the south of the north temperate zone, although it presents a continental nature due to its mean elevation (700 ma.s.l.) and distance from sea. Farmland is the most common type of land use in the aquifer. The most limiting factor for farming is the weather. This area is classified as semiarid (aridity index (AI) 0.26) [39]. Annual reference evapotranspiration values (ETo) are from 1165 mm year−1 in the central area of the aquifer to more than 1300 mm year−1 in the northwest and southeast. Agro-climatic stations showed precipitation values from 336 to 413 mm year−1 with a maximum value of 82 mm in summertime. The analysis of thermal characteristics shows variations from 19.3 to 20.8 °C for annual mean daily maximum temperature and from 6.3 to 6.6 °C for annual mean daily minimum temperatures [40]. These constraints determine the following four groups of crops with different culture systems [41]: (1) cold weather crops that are sown in the autumn or early winter and are harvested at the end of spring or at the beginning of summer;(2) warm season crops for which the growing cycle develops in the summer;(3) rain-fed crops that are limited by the rainfall regimes and (4) irrigated crops. As an example, the crop distribution in 2012 included 14.2% wheat, 14.0% barley, 7.1% maize, 5.8% woody crops, 5.6% opium, 4.9% garlic, 4.4% alfalfa and 3.9% onion, garlic, pea, double crops and other vegetables [42]. The size of the irrigated farms is also important. Farms with more than 100 ha of irrigated area account for 32% of the total area. A significant portion of the total water is consumed by large farms, which are generally equipped with high performance irrigation systems. In addition, collective irrigated farms that were created by public initiative and organized into Water Users Associations (WUAs) are included within the group of farms with a surface area of more than 100 ha. The main irrigation methods include permanent solid set systems (39.2%), center pivot systems (37.8%), drip irrigation systems (17.7%), surface irrigation systems (3.6%) and portable sprinkler systems (1.7%) [43]. After significant efforts were made during the modernization process, farmers demanded management systems and tools to obtain the potential water and energy use efficiency that can be provided by these irrigation systems. In this sense, irrigation has become essential for rural development and maintenance.

2.3. Ground Truth Data

The field visits to obtain the ground truth were performed by the Confederación Hidrográfica del Júcar, Spain (www.chj.es) during the irrigation season of 2016. They visited 6341 plots that covered 28,963 ha. After the spatial analysis of the plots, which eliminated plots with a size area less than 1 ha, the number of used plots was reduced to 3111 (24,208 ha). Finally, after the selection of the crops of interest for this study (Table 2), the number of utilized plots was 2032, representing an area of 17,281 ha, which covers 15% of the irrigated area in the Mancha Oriental Aquifer. The preparation of ground truth data was conducted using QGIS software.
A selection of the main crops (12) established in the area was analyzed. The double crops were not included, although they are relatively abundant, because the field visits were performed for each plot only once a year. These are easily detectable plots due to the special pattern of the NDVI during the irrigation season. Thus, it will not have a negative effect on the results.
To illustrate the NDVI pattern of some studied crops during the period march–July, wepresented in Figure 3 the NDVI temporal behavior of six crops from the selected twelve.

2.4. Remote Sensing Information and Processing

Using a parcel-based approach [44], the average NDVI value and standard deviation for each plot were calculated. Only plots larger than 1 ha were utilized to ensure that there was a sufficient number of pixels inside each plot (3 pixels × 3 pixels for Landsat-8). To select pixels that were completely inside the plot and thus avoid a border effect, a buffer of 30 m inside the plot was implemented. With all the pixels inside the plot that fulfill these conditions, average and standard deviation values of the NDVI were obtained. To determine the threshold value of standard deviation an analysis of these values for two representative dates (May 6th and August 1st) was analyzed. Standard deviation values of all the plots for both dates were sorted and plotted. The values reached the asymptote for an approximate value of 0.1 in both cases, establishing this value as threshold. The observed variations at the plot level could be explained by the fact that the farmer did not grow the whole plot due to the effect of some pest or weed or because of improper management. These plots were eliminated to ensure a proper ground truth.
The images detailed in Table 3 were utilized in this case study. Additionally, Table 3 shows the percentage of cloud cover of each image. Cloud cover is a key aspect to consider when assessing crop classification with satellite-based remote sensing techniques [45]. The criteria for selecting a maximum cloud cover percentage is heterogeneous, from 10% [46] to a case in which cloud cover percentage was not considered [47]. Among those criteria, we found references that considered 20% [19], 25% [29,48] and even 60% [13]. In this context of nonstandardized criteria, we selected 40%, which resulted in the elimination of only three images from the set. This is a key issue that should be investigated in areas where cloud cover is more intense (humid or sub-humid regions). To avoid this effect, SAR (Synthetic Aperture Radar) information can contribute in areas were the presence of clouds limits the applicability of the proposed methodology [49].

2.5. Analysis of Interoperability between Landsat-8 and Sentinel 2

Although all images were utilized to carry out crop classification, to perform the interoperability analysis between Landsat-8 and Sentinel-2A, only three dates when both satellites covered the case study in the same day were utilized (24 May, 12 August and 31 October). The first date was coincided with maximum vegetation cover of the spring crops, the second date with maximum vegetation cover of summer crops and the last date with senescence of summer crops and orchards. The area included bare soils and crop cover to evaluate the widest range of NDVI values. Figure 4 shows the difference between the NDVI Landsat-8 and Sentinel-2A for the three dates analyzed for a spring crop (two pivots on the east) and a summer crop (two pivots on the west) as an example. On 24 May the spring crops was close to full cover, on 12 August the summer crop was also at this stage, and on 31 October all the pivots were without crops. It can be seen that the difference in NDVI values among both sources of information (Landsat-8 and Sentinel-2A) were higher (blue color) when vegetation is in the low development stage or not vegetating than when the crop was close to full cover. Thus, both sources could be interoperable in the case of the high presence of vegetation, but there is a mismatch among NDVI values of both satellites when there is low crop cover.
To perform the analysis of interoperability, a grid of points separated 30 m in the analyzed area was generated. In each point of this mesh, the Landsat-8 NDVI value was calculated. In the case of Sentinel-2A, the bands of the nine neighboring pixels were averaged and the NDVI value was calculated from these mean bands. Once the data to evaluate the interoperability between the two sources was obtained, a statistical analysis was performed by calculating the coefficient of determination (R2), the root mean square error (RMSE), and the relative error (RE). The normality and homoscedasticity of the residuals were also analyzed. A calibration equation for the area was obtained to interoperate with both sources of data.
The results of the comparison between NDVI values of Landsat-8 and Sentinel-2A for the three dates (24 May, 12 August and 31 October) and the area analyzed (Figure 4) are shown in Figure 5.
There is an underestimation of NDVI values obtained with Sentinel-2A compared with those obtained with Landsat-8 for all the dates and the whole data set. However, the dispersion of the data is low, which means that there is a good correlation between both sources of data (Table 4).
The difference between both sources of data fits a normal distribution and the residuals are homoscedastic. The main statistics of the relations between both datasets are shown in Table 4.
With these results, it can be concluded that a linear calibration can be obtained to ensure the interoperability between both sources of data. We decided to correct Landsat-8 values to fit Sentinel-2A NDVI values with the Equation (1).
NDVI-S2 = −0.085 + 1.0759·NDVI-L8,

2.6. Classification Methods

A set of 22 classification algorithms was evaluated comprising of decision trees, discriminant analysis, support vector machines, nearest neighbor and ensemble classifiers. These algorithms are the most frequent algorithms used in the literature review summarized in the Introduction Section. Table 5 describes the main characteristics of each method. The Classification Learner application of Matlab® was utilized with the aim of calibrating and validating the different algorithms. The Classification Learner app, can be found in the latest versions of the Statistics and Machine Learning toolbox of Matlab. This application enables automated training for the set of supervised machine learning classifiers incorporated in the toolbox. Besides, trained classifiers can be exported to the Matlab workspace. With Matlab software, the app supports a total of 22 classifier types, organized in five major classification algorithms: decision trees, discriminant analysis, support vector machines, nearest neighbor and ensemble classifiers. Additionally, the Classification Learner app offers built-in validation schemes that point out the predictive accuracy of the trained model. The objective of this manuscript was to define the best performing classifier group in crop classification. Thus, the default hyperparameters for each classifier were utilized. In future works, refinement of these hyperparameters will be performed. A detailed description of each of the classifier groups can be found in the basic literature treating about machine learning algorithms.
The input information for each of the classification algorithms are the NDVI values for each available date (Table 3) and the output information is a label corresponding to each crop type. Cross-validation with 5 folds was carried out for each algorithm in the calibration process to avoid overfitting problems.

2.7. Evaluation of Classification Accuracy

A confusion matrix and classical performance indicators of the overall accuracy, producer’s accuracy and user’s accuracy were obtained. The kappa statistic, which is traditionally used to test the classification accuracy, presents several limitations in accuracy testing [50]. Additionally, the different number of plots for each crop can make kappa values misinterpret the obtained results. Thus, weighted F1 score statistic was used for classification algorithm selection [51] and this accuracy metrics was calculated using a Python library called Scikit-Learn. Once the most accurate algorithm for each dataset has been selected, the remainder indicators, together with the confusion matrix, were analyzed to determine the efficiency in the classification for each crop and group of crops.

3. Results

3.1. Evaluation of Classification Methods for Crop Classification

An analysis of the available images that cover the case study for Landsat-8 and Sentinel-2A was performed. The results of using only Landsat-8 information, only Sentinel-2A information and a combination of both sources of information were obtained for the evaluated classification methods. A comparison between the 22 classification methods with the different combinations of available information is shown in Table 6.
The method with the highestF1 score value for all of the instances of available information was the subspace ensemble method, with nearest neighbor learners (M21). Sentinel-2A information slightly improved the F1 score value of the classification compared with using only Landsat-8 information (from 0.86 to 0.87). The use of Sentinel-2A or Landsat-8 independently returned almost the same results. Additionally, ensuring interoperability between both sources of information did not provide a remarkable improvement in the classification results. This fact might be due to the similarities between Landsat-8 and Sentinel-2A for high NDVI values, as stated in the interoperability analysis above performed. Additionally, to perform a proper crop classification, only NDVI patterns were requested while absolute NDVI values were not required. Thus, improving NDVI values with the interoperability procedure did not contribute to improve crop classification.
Among the groups of classification methods, decision tress, discriminant analysis, support vector machines, nearest neighbor classifiers and ensemble classifiers, the second-best classification method for combined information from Landsat-8 and Sentinel-2A was the support vector machine with quadratic kernel (M7) with a F1 score = 0.88, and the fine KNN (k-nearest neighbor), (M12) with F1 score = 0.88. Decision trees, which are commonly used in classification, returned poor results, with a F1 score = 0.76 for the best method (complex tree). The discriminant analysis was not adequate for classification issues resulting in F1 score values ranging between 0.76 and 0.66. The comparison between the prediction speed (expressed in observation per second) and the classification (F1 Score) is illustrated in Figure 6.

3.2. Selection and Evaluation of the Best Method for Crop Classification

To select the best classification method, one should consider the maximum efficiency (maximum F1 score) or a combination of high prediction speed and high accuracy (not the highest) shown in Figure 6. Thus, the subspace ensemble method with nearest neighbor learners (M21) as the most accurate method and the nearest neighbor classifier with fine KNN (M12) as the best agreement or combination between accuracy and prediction speed were evaluated. It is important to mention, that definitely the classifier with the highest accuracy (M21 in this case) had the highest priority to be selected as the best method for crop classification. To evaluate the results of the methods, the confusion matrix and the producer’s and user’s accuracies (PA and UA) were calculated (Table 7 and Table 8).
Crops with UA of 100% means that all the testing plots with this crop were properly classified, which in this case was poppy. Crops with a UA higher than 95% were almond trees (96.8%), with confusion with vineyard and maize (95.9%) and confusion with onion and sunflower (there was one plot of barley that it was probably due to mistakes in the field data). Purple garlic (UA = 93.1%) was differentiated from white garlic (93.3) even when their phenological cycle was close. Purple garlic plots were classified in one plot as white garlic and in another plot as peas. Barley (UA = 82.3%) was confused with wheat for 13 plots, white garlic for four plots, and peas for six plots, with these crops having similar cycles.
When analyzing the results of the M12 classification method, which has a relatively high prediction speed with a high accuracy, the UA and PA were almost the same as M21. Only slight changes in onion, sunflower, vineyard, peas (one plot misclassified), purple garlic and almond trees (two plots misclassified) were observed. Additionally, some other plots were better classified, such as barley and wheat (one plot improved). Thus, the combined analysis of F1 score value and the confusion matrix was determinant to select the most appropriate algorithm.
In the case of sunflower, there were only 41 plots for ground truth, being one of the crops with a lower number of plots visited. This fact led to a worse calibration and testing of the algorithms, with UA values of 69.2% (the lowest value). Additionally, this crop had different types of management depending upon the farmer. Sowing dates can range from early April to late July, which could lead to a misclassification. The 2016 irrigation season presented a cold spring in the area, which delayed the sowing date of the onion. This fact could have led the algorithm to confuse the sunflower with the onion primarily because of the similarities in the initial stages of these crops. This fact would also justify the high PA value for onion (90%), because the main problem detected is that some onion plots were classified as sunflower. Additionally, sunflower is not usually managed to produce the maximum yield to reduce irrigation and fertilization costs, making the development of the sunflower closer to that of the onion.
Another crop with a low number of visited plots was peas, with 98 visited plots and a UA = 70%. For peas, the objectivized yield and growing cycles are heterogeneous. We could find proteaginous peas with quite different cycles than the green peas grown for industry. This heterogeneity of growing cycles and the climatological conditions of 2016, which presented severe frost in spring, justify the confusion between peas, barley and wheat.
It deserves highlighting the high UA for poppy (100%), which presents a similar cycle to barley and wheat. This could be due to the fact that is a crop totally controlled by the pharmacy industry and they homogenize the varieties, cycles, and management, leading to the proper classification. Maize and almonds have a UA higher than 95% without any tendency towards misclassification, probably due to mistakes during the field visits works.

4. Discussion

In this paper, we aimed to test a total of 22 nonparametric classification algorithms, organized in five major classification algorithms: decision trees, discriminant analysis, support vector machines, nearest neighbor and ensemble classifiers. Results showed that among these 22 classifiers the subspace ensemble method with nearest neighbor learners (M21) and the nearest neighbor classifier with fine KNN (M12) were the best performing methods based on two criteria: high accuracy of the F1 score and high prediction speed. However, as mentioned previously in the Results Section, accuracy was the most important criteria. Indeed previous research has indicated that the integration of two or more classifiers, which is the case of (M21), improve the classification accuracy compared to the use of a single classifier [31,52,53]. Focusing on the results of the other classifiers, we noticed that the decision trees compared to the rest of the algorithms did not provide the most accurate results (the average F1 score value obtained for decision trees in our analysis was equal to 0.68, while the best classifier obtained a F1 score equal to 0.87), but did provide the highest prediction speed ((22000, 20000 and 20000 observation/second respectively for M2, M1 and M3 compared with the best classifier (M21) that resulted in 710 observations/second). In the literature, decision trees classifiers has been widely used in classification process [3] for the just reason that it is known as the speediest classifiers that can be trained [54].According to [53,55], SVM has been considered as a powerful technique for classification tasks, but unexpectedly in this work the coarse Gaussian SVM classifiers (M11) performed less than ensemble classifiers such as (M1, M19 and M21).
Generally the use of only Sentinel-2A data provideda slightly higher F1 score than the use of Landsat-8, and, the difference is barely noticeable ranging between 1% and 3%. However for some classifier we observed that the classification with Sentinel-2A data yielded better results (F1 score equal or higher than 4%) than with Landsat-8, such as, (M2, M4, M1, M13, M14, M16 and M20). Regarding the synergic data use, this methodology brought only slight improvement when the following classifiers were tested (M1, M4, M6, M7, M8, M10, M12, M18, M19, M20, M21 and M22), but for the rest of them we observed an F1 score equal (M3 and M11) or even lower (M1, M2, M5, M9, M13, M14, M15, M16 and M17) than the results obtained with only usingSentinel-2A data. Thus, with Sentinel-2A data the majority of the classifiers presented a slightly higher F1 score and when we used both data sources we did not notice any outstanding improvement in the classification results. Consequently, it can be concluded that it is not worthy to include Landsat-8 data especially sinceSentinel-2A has higher spatial and temporal resolution.
Additionally, this study evaluated the interoperability between Sentinel-2A and Landsat-8 through the use of ToA NDVI values. Atmospheric corrections are not necessary in this application, avoiding an intensive workload. Indeed, in the work conducted by [31], the classification was performed using Sentinel-2 (Level-1C) and Landsat-8 (Level-1T) and the results showed an overall accuracy ranging between 87% and 92%, when classifying crops individually (15 classes) and when crops were grouped based on phenology, prior to classification (seven classes), respectively. Additionally, for the classification process developed by [14], the authors used Sentinel 2 (ToA) product. We obtained adequate interoperability results. This suitability can be explained by the fact that the NDVI normalizes the information obtained from both sensors by reducing the divergence between their spectral bands. However, underestimation of NDVI values was observed when Sentinel-2A was compared with Landsat-8. First of all, at the beginning of the crop cycles when NDVI values are low (at the start of the growing season the soil was not yet covered by the vegetation), but also when NDVI values were high (when the vegetation cover reach the maximum). This matter could have a greater impact on other applications, such as the estimation of water volumes from NDVI values but for classification application, we just need the tendency of the NDVI values along time.

5. Conclusions

The combined use of Sentinel-2A and Landsat-8 information did not greatly improve crop classification results compared with using only Landsat-8or only Sentinel-2A. Incorporating interoperability between Landsat-8 and Sentinel-2A information only slightly improved classification. However, in areas where cloud cover is more frequent, having a higher temporal resolution would lead to a better performance of Sentinel-2A over Landsat-8. In addition, the higher spatial resolution of Sentinel-2A compared with Landsat-8 permits classifying smaller plots because the buffer to be utilized would be 10 m instead of 30 m, as is used for Landsat-8 to avoid a border effect.
Between the classification algorithms utilized, the most robust algorithm was the subspace ensemble method, with nearest neighbor learners (M21) followed by (M12) the nearest neighbor classifier with fine KNN as the method with the best balance between accuracy and processing time. F1 score values can give a general overview of method performance, but it is necessary to evaluate the confusion matrix with an agronomic perspective to conclude the best method to perform crop classification.
The main conclusion was obtained with the spring crops, the crop cycles of which are close in time and of which variability was dependent on the sowing date and climatic conditions. Woody crops and summer crops were easier to distinguish in semiarid regions because of their difference in the growing cycles.
The use of optical remotes sensing data can be limited by the presence of clouds that introduce missing values in the dataset affecting the capturing of the seasonality of the vegetation cover. For this reason, for future studies we will focus on merging radar with optical data especially with the availability of Sentinel 1 data, which have the same spatial resolution of Sentinel 2. Indeed, several studies ensures that the synergic use of optical and radar data improve the classification OA [56,57,58]. Similarly the use of the data of this work along with radar data can be used to extract valuable phenological parameters to map the crop type behavior of the study area as performed in [59].

Author Contributions

Conceptualization, D.H.-L., R.B. and M.A.M.; Methodology, A.C., D.O.-T., D.H.-L., R.B., J.F.O. and M.A.M.; Software, D.O.-T., D.H.-L. and M.A.M.; Validation, D.H.-L. and M.A.M.; Visualization, D.O.-T.; Formal analysis, A.C., D.H.-L., R.B., J.F.O. and M.A.M.; Investigation, R.B. and M.A.M.; Data curation, A.C. and D.O.-T.; Writing—original draft, A.C., R.B. and M.A.M.; Writing—review and editing, R.B., J.F.O. and M.A.M.; Visualization, D.O.-T.; Supervision, D.H.-L.; Project administration, J.F.O.; Funding acquisition, D.H.-L. and M.A.M. All authors have read and agreed to the published version of the manuscript.

Funding

“This research was funded by Spanish Ministry of Education and Science (MEC) grant number AGL2017-82927-C3-2-R (Co-funded by FEDER)”.

Acknowledgments

We would like to acknowledge the support of the Confederación Hidrográfica del Júcar. We would also acknowledge the precious help of Juan Luis (UCLM sport service) and Carmen (UCLM cleaning service) for making possible the access to the data in times of COVID-19.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Murmu, S.; Biswas, S. Application of Fuzzy Logic and Neural Network in Crop Classification: A Review. Aquat. Procedia 2015, 4, 1203–1210. [Google Scholar] [CrossRef]
  2. Durgun, Y.Ö.; Gobin, A.; Van De Kerchove, R.; Tychon, B. Crop area mapping using 100-m Proba-V time series. Remote Sens. 2016, 8, 585. [Google Scholar] [CrossRef] [Green Version]
  3. Lu, D.; Weng, Q. Review article A survey of image classification methods and techniques for improving classification performance. Int. J. Remote Sens. 2007, 28, 823–870. [Google Scholar] [CrossRef]
  4. Duro, D.C.; Franklin, S.E.; Dubé, M.G. A comparison of pixel-based and object-based image analysis with selected machine learning algorithms for the classification of agricultural landscapes using SPOT-5 HRG imagery. Remote Sens. Environ. 2012, 118, 259–272. [Google Scholar] [CrossRef]
  5. Ok, A.O.; Akar, O.; Gungor, O. Evaluation of random forest method for agricultural crop classification. Eur. J. Remote Sens. 2017, 7254. [Google Scholar] [CrossRef]
  6. Peña, J.M.; Gutiérrez, P.A.; Hervás-Martínez, C.; Six, J.; Plant, R.E.; López-Granados, F. Object-based image classification of summer crops with machine learning methods. Remote Sens. 2014, 6, 5019–5041. [Google Scholar] [CrossRef] [Green Version]
  7. Raczko, E.; Zagajewski, B. Comparison of support vector machine, random forest and neural network classifiers for tree species classification on airborne hyperspectral APEX images. Eur. J. Remote Sens. 2017, 50, 144–154. [Google Scholar] [CrossRef] [Green Version]
  8. Atzberger, C.; Rembold, F. Mapping the spatial distribution of winter crops at sub-pixel level using AVHRR NDVI time series and neural nets. Remote Sens. 2013, 5, 1335–1354. [Google Scholar] [CrossRef] [Green Version]
  9. Warrender, C.E.; Augusteijn, M.F. Fusion of image classifications using bayesian techniques with markov random fields. Int. J. Remote Sens. 1999, 20, 1987–2002. [Google Scholar] [CrossRef]
  10. Ozdogan, M.; Yang, Y.; Allez, G.; Cervantes, C. Remote Sensing of Irrigated Agriculture: Opportunities and Challenges. Remote Sens. 2010, 2, 2274–2304. [Google Scholar] [CrossRef] [Green Version]
  11. Schmedtmann, J.; Campagnolo, M.L. Reliable crop identification with satellite imagery in the context of Common Agriculture Policy subsidy control. Remote Sens. 2015, 7, 9325–9346. [Google Scholar] [CrossRef] [Green Version]
  12. Yusoff, N.M.; Muharam, F.M. The Use of Multi-Temporal Landsat Imageries in Detecting Seasonal Crop Abandonment. Remote Sens. 2015, 7, 11974–11991. [Google Scholar] [CrossRef] [Green Version]
  13. Schmidt, M.; Pringle, M.; Devadas, R.; Denham, R.; Tindall, D. A Framework for Large-Area Mapping of Past and Present Cropping Activity Using Seasonal Landsat Images and Time Series Metrics. Remote Sens. 2016, 8, 312. [Google Scholar] [CrossRef] [Green Version]
  14. Immitzer, M.; Vuolo, F.; Atzberger, C. First experience with Sentinel-2 data for crop and tree species classifications in central Europe. Remote Sens. 2016, 8, 166. [Google Scholar] [CrossRef]
  15. Mandanici, E.; Bitelli, G. Preliminary comparison of sentinel-2 and landsat 8 imagery for a combined use. Remote Sens. 2016, 8, 1014. [Google Scholar] [CrossRef] [Green Version]
  16. Addabbo, P.; Focareta, M.; Marcuccio, S.; Votto, C.; Ullo, S.L. Contribution of Sentinel-2 data for applications in vegetation monitoring. Acta IMEKO 2016, 5, 44–54. [Google Scholar] [CrossRef]
  17. Li, J.; Roy, D.P. A global analysis of Sentinel-2a, Sentinel-2b and Landsat-8 data revisit intervals and implications for terrestrial monitoring. Remote Sens. 2017, 9, 902. [Google Scholar] [CrossRef] [Green Version]
  18. Roy, D.; Wulder, M.A.; Loveland, T.R.; Woodcock, C.E.; Allen, R.G.; Anderson, M.; Helder, D.; Irons, J.; Johnson, D.; Kennedy, R.; et al. Landsat-8: Science and product vision for terrestrial global change research. Remote Sens. Environ. 2014, 145, 154–172. [Google Scholar] [CrossRef] [Green Version]
  19. Azar, R.; Villa, P.; Stroppiana, D.; Crema, A.; Boschetti, M.; Brivio, P.A.; Azar, R.; Villa, P.; Stroppiana, D.; Crema, A. Assessing in-season crop classification performance using satellite data: A test case in Northern Italy Assessing in-season crop classification performance. Eur. J. Remote Sens. 2017, 7254. [Google Scholar] [CrossRef] [Green Version]
  20. Sonobe, R.; Yamaya, Y.; Tani, H.; Wang, X.; Kobayashi, N.; Mochizuki, K. Crop classification from Sentinel-2-derived vegetation indices using ensemble learning. J. Appl. Remote Sens. 2018, 12, 026019. [Google Scholar] [CrossRef] [Green Version]
  21. Villa, P.; Bresciani, M.; Braga, F.; Bolpagni, R. Comparative assessment of broadband vegetation indices over aquatic vegetation. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2014, 7, 3117–3127. [Google Scholar] [CrossRef]
  22. Tucker, C.J. Red and photographic infrared linear combinations for monitoring vegetation. Remote Sens. Environ. 1979, 8, 127. [Google Scholar] [CrossRef] [Green Version]
  23. Sesnie, S.E.; Gessler, P.E.; Finegan, B.; Thessler, S. Integrating Landsat TM and SRTM-DEM derived variables with decision trees for habitat classification and change detection in complex neotropical environments. Remote Sens. Environ. 2008, 112, 2145–2159. [Google Scholar] [CrossRef]
  24. Achard, F.; Blasco, F. Analysis of vegetation seasonal evolution and mapping of forest cover in West Africa with the use of NOAA AVHRR HRPT data. Photogramm. Eng. Remote Sens. 1990, 56, 1359–1365. [Google Scholar]
  25. Ustuner, M.; Sanli, F.B.; Dixon, B. Application of Support Vector Machines for Landuse Classification Using High-Resolution RapidEye Images: Application of Support Vector Machines for Landuse Classification Using High-Resolution RapidEye Images: A Sensitivity Analysis. Eur. J. Remote Sens. 2017, 7254. [Google Scholar] [CrossRef]
  26. Inglada, J.; Arias, M.; Tardy, B.; Hagolle, O.; Valero, S.; Morin, D.; Dedieu, G.; Sepulcre, G.; Bontemps, S.; Defourny, P.; et al. Assessment of an operational system for crop type map production using high temporal and spatial resolution satellite optical imagery. Remote Sens. 2015, 7, 12356–12379. [Google Scholar] [CrossRef] [Green Version]
  27. Tatsumi, K.; Yamashiki, Y.; Angel, M.; Torres, C.; Leonidas, C.; Taipe, R. Crop classification of upland fields using Random forest of time-series Landsat 7 ETM + data. Comput. Electron. Agric. 2015, 115, 171–179. [Google Scholar] [CrossRef]
  28. Blaschke, T.; Hay, G.J.; Kelly, M.; Lang, S.; Hofmann, P.; Addink, E.; Queiroz Feitosa, R.; van der Meer, F.; van der Werff, H.; van Coillie, F.; et al. Geographic Object-Based Image Analysis-Towards a new paradigm. ISPRS J. Photogramm. Remote Sens. 2014, 87, 180–191. [Google Scholar] [CrossRef] [Green Version]
  29. Schultz, B.; Immitzer, M.; Formaggio, A.R.; Del, I.; Sanches, A.; José, A.; Luiz, B.; Atzberger, C. Self-Guided Segmentation and Classification of Multi-Temporal Landsat 8 Images for Crop Type Mapping in Southeastern Brazil. Remote Sens. 2015, 7, 14482–14508. [Google Scholar] [CrossRef] [Green Version]
  30. Heupel, K.; Spengler, D.; Itzerott, S. A Progressive Crop-Type Classification Using Multitemporal Remote Sensing Data and Phenological Information. PFG J. Photogramm. Remote Sens. Geoinf. Sci. 2018, 86, 53–69. [Google Scholar] [CrossRef] [Green Version]
  31. Piedelobo, L.; Hernández-López, D.; Ballesteros, R.; Chakhar, A.; Del Pozo, S.; González-Aguilera, D.; Moreno, M.A. Scalable pixel-based crop classification combining Sentinel-2 and Landsat-8 data time series: Case study of the Duero river basin. Agric. Syst. 2019, 171, 36–50. [Google Scholar] [CrossRef]
  32. Belgiu, M.; Csillik, O. Sentinel-2 cropland mapping using pixel-based and object-based time-weighted dynamic time warping analysis. Remote Sens. Environ. 2018, 204, 509–523. [Google Scholar] [CrossRef]
  33. Congalton, R.G. A Review of Assessing the Accuracy of Classification of Remotely Sensed Data A Review of Assessing the Accuracy of Classifications of Remotely Sensed Data. Remote Sens. Environ. 1991, 4257, 34–46. [Google Scholar]
  34. Hao, P.; Tang, H.; Chen, Z.; Yu, L.; Wu, M. High resolution crop intensity mapping using harmonized Landsat-8 and Sentinel-2 data. J. Integr. Agric. 2019, 18, 2883–2897. [Google Scholar] [CrossRef]
  35. Orynbaikyzy, A.; Gessner, U.; Conrad, C. Crop type classification using a combination of optical and radar remote sensing data: A review. Int. J. Remote Sens. 2019, 40, 6553–6595. [Google Scholar] [CrossRef]
  36. Kobayashi, N.; Tani, H.; Wang, X.; Sonobe, R. Crop classification using spectral indices derived from Sentinel-2A imagery. J. Inf. Telecommun. 2019, 4, 67–90. [Google Scholar] [CrossRef]
  37. Htitiou, A.; Boudhar, A.; Lebrini, Y.; Hadria, R.; Lionboui, H.; Elmansouri, L.; Tychon, B.; Benabdelouahab, T. The Performance of Random Forest Classification Based on Phenological Metrics Derived from Sentinel-2 and Landsat 8 to Map Crop Cover in an Irrigated Semi-arid Region. Remote Sens. Earth Syst. Sci. 2019, 2, 208–224. [Google Scholar] [CrossRef]
  38. Nguyen, M.D.; Baez-Villanueva, O.M.; Bui, D.D.; Nguyen, P.T.; Ribbe, L. Harmonization of Landsat and Sentinel 2 for Crop Monitoring in Drought Prone Areas: Case Studies of Ninh Thuan (Vietnam) and Bekaa (Lebanon). Remote Sens. 2020, 12, 281. [Google Scholar] [CrossRef] [Green Version]
  39. United Nations Environment Programme. UNEP World Atlas of Desertification, 2nd ed.; Wiley: Hoboken, NJ, USA, 1997; p. 182. [Google Scholar]
  40. Ballesteros, R.; Ortega, J.F.; Moreno, M.Á. FORETo: New software for reference evapotranspiration forecasting. J. Arid Environ. 2016, 124, 128–141. [Google Scholar] [CrossRef]
  41. De Juan, J.A.; Ortega, J.F.; Tarjuelo, J.M. Sistemas de Cultivo: Evaluación de Itinerarios Técnicos. (Farming Systems: Culture Assessments); Mundi-Prensa Libros: Madrid, Spain, 2003. (In Spanish) [Google Scholar]
  42. Junta Central de Regantes de la Mancha Oriental. Available online: http://www.jcrmo.org/wp-content/uploads/2018/01/Memoria_2016.pdf (accessed on 10 December 2019).
  43. Junta Central de Regantes de la Mancha Oriental. Available online: http://www.jcrmo.org/wp-content/uploads/2018/01/MEMORIA_2014_COMPLETA_reducida.pdf (accessed on 10 December 2019).
  44. Hermosilla, T.; Díaz-Manso, J.M.; Ruiz, L.A.; Recio, J.A.; Fernández-Sarría, A.; Ferradáns-Nogueira, P. Parcel-based image classification as a decision-making supporting tool for the land bank of Galicia (Spain). Core Spat. Databases—Updat. Maint. Serv.—From Theory to Pract. 2010, 38, 42–45. [Google Scholar]
  45. Eberhardt, I.D.R.; Schultz, B.; Rizzi, R.; Sanches, I.D.A.; Formaggio, A.R.; Atzberger, C.; Mello, M.P.; Immitzer, M.; Trabaquini, K.; Foschiera, W.; et al. Cloud cover assessment for operational crop monitoring systems in tropical areas. Remote Sens. 2016, 8, 219. [Google Scholar] [CrossRef] [Green Version]
  46. Hao, P.; Wang, L.; Niu, Z.; Aablikim, A.; Huang, N.; Xu, S.; Chen, F. The Potential of Time Series Merged from Landsat-5 TM and HJ-1 CCD for Crop Classification: A Case Study for Bole and Manas Counties in Xinjiang, China. Remote Sens. 2014, 6, 7610–7631. [Google Scholar] [CrossRef] [Green Version]
  47. Inglada, J.; Vincent, A.; Arias, M.; Marais-Sicre, C. Improved Early Crop Type Identification By Joint Use of High Temporal Resolution SAR And Optical Image Time Series. Remote Sens. 2016, 8, 362. [Google Scholar] [CrossRef] [Green Version]
  48. Eggen, M.; Ozdogan, M.; Zaitchik, B.F.; Simane, B. Land cover classification in complex and fragmented agricultural landscapes of the Ethiopian highlands. Remote Sens. 2016, 8, 1020. [Google Scholar] [CrossRef] [Green Version]
  49. Sun, Y.; Luo, J.; Wu, T.; Zhou, Y.; Liu, H.; Gao, L.; Dong, W.; Liu, W.; Yang, Y.; Hu, X.; et al. Synchronous Response Analysis of Features for Remote Sensing Crop Classification Based on Optical and SAR Time-Series Data. Sensors 2019, 19, 4227. [Google Scholar] [CrossRef] [Green Version]
  50. Foody, G.M. Explaining the unsuitability of the kappa coefficient in the assessment and comparison of the accuracy of thematic maps obtained by image classification. Remote Sens. Environ. 2020, 239. [Google Scholar] [CrossRef]
  51. Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; et al. Scikit-learn: Machine Learning in Python. J. Mach. Learn. Res. 2011, 12, 2825–2830. [Google Scholar]
  52. Du, P.; Xia, J.; Zhang, W.; Tan, K.; Liu, Y.; Liu, S. Multiple classifier system for remote sensing image classification: A review. Sensors 2012, 12, 4764–4792. [Google Scholar] [CrossRef]
  53. Chen, Y.; Dou, P.; Yang, X. Improving land use/cover classification with a multiple classifier system using AdaBoost integration technique. Remote Sens. 2017, 9, 1055. [Google Scholar] [CrossRef] [Green Version]
  54. Choodarathnakara, A.L.; Kumar, D.T.A.; Koliwad, D.S.; Patil, D.C.G. Mixed Pixels: A Challenge in Remote Sensing Data Classification for Improving Performance. Int. J. Adv. Res. Comput. Eng. Technol. 2012, 1, 261–271. [Google Scholar]
  55. Khatami, R.; Mountrakis, G.; Stehman, S.V. A meta-analysis of remote sensing research on supervised pixel-based land-cover image classification processes: General guidelines for practitioners and future research. Remote Sens. Environ. 2016, 177, 89–100. [Google Scholar] [CrossRef] [Green Version]
  56. Van Tricht, K.; Gobin, A.; Gilliams, S.; Piccard, I. Synergistic use of radar sentinel-1 and optical sentinel-2 imagery for crop mapping: A case study for Belgium. Remote Sens. 2018, 10, 1642. [Google Scholar] [CrossRef] [Green Version]
  57. Kussul, N.; Mykola, L.; Shelestov, A.; Skakun, S. Crop inventory at regional scale in Ukraine: Developing in season and end of season crop maps with multi-temporal optical and SAR satellite imagery. Eur. J. Remote Sens. 2018, 51, 627–636. [Google Scholar] [CrossRef] [Green Version]
  58. Demarez, V.; Helen, F.; Marais-Sicre, C.; Baup, F. In-season mapping of irrigated crops using Landsat 8 and Sentinel-1 time series. Remote Sens. 2019, 11, 118. [Google Scholar] [CrossRef] [Green Version]
  59. Stendardi, L.; Karlsen, S.R.; Niedrist, G.; Gerdol, R.; Zebisch, M.; Rossi, M.; Notarnicola, C. Exploiting time series of Sentinel-1 and Sentinel-2 imagery to detect meadow phenology in mountain regions. Remote Sens. 2019, 11, 542. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Flowchart of the methodology applied for crop classification.
Figure 1. Flowchart of the methodology applied for crop classification.
Remotesensing 12 01735 g001
Figure 2. Plot visited for obtaining the ground truth. On the top-right, it is show the case study located in Spain. To the left, all the visited plots distributed across the area (green plots). At the bottom-right part, it is shown several plots after spatial analysis is implemented (buffer 30 m and eliminating plot with S < 1 ha; CRS-EPSG: 25830).
Figure 2. Plot visited for obtaining the ground truth. On the top-right, it is show the case study located in Spain. To the left, all the visited plots distributed across the area (green plots). At the bottom-right part, it is shown several plots after spatial analysis is implemented (buffer 30 m and eliminating plot with S < 1 ha; CRS-EPSG: 25830).
Remotesensing 12 01735 g002
Figure 3. Normalized difference vegetation index (NDVI) temporal features for the period March-July for poppy, purple garlic, barley, vineyard, maize and peas.
Figure 3. Normalized difference vegetation index (NDVI) temporal features for the period March-July for poppy, purple garlic, barley, vineyard, maize and peas.
Remotesensing 12 01735 g003
Figure 4. Difference between Landsat-8 and Sentinel-2A NDVI values for 4 pivots in three dates (24 May, 12 August and 31 October). The two pivots on the East side had a spring crop, and the two on the west side had a summer crop.
Figure 4. Difference between Landsat-8 and Sentinel-2A NDVI values for 4 pivots in three dates (24 May, 12 August and 31 October). The two pivots on the East side had a spring crop, and the two on the west side had a summer crop.
Remotesensing 12 01735 g004
Figure 5. Comparison between NDVI values obtained with Landsat-8 and Sentinel-2A images for (a) May 24th, (b) August 12th, (c) October 31st and (d) the whole data set.
Figure 5. Comparison between NDVI values obtained with Landsat-8 and Sentinel-2A images for (a) May 24th, (b) August 12th, (c) October 31st and (d) the whole data set.
Remotesensing 12 01735 g005aRemotesensing 12 01735 g005b
Figure 6. Prediction speed in observations per second compared with classification accuracy (F1 score).
Figure 6. Prediction speed in observations per second compared with classification accuracy (F1 score).
Remotesensing 12 01735 g006
Table 1. Summary of some studies about crop classification.
Table 1. Summary of some studies about crop classification.
StudyLocationNumber of Classified CropsSensorNumber of ImagesClassification AlgorithmOverall Accuracy
[14]Austria and Germany7S21Random Forest76%
[19]Italy7L8Up to 13Maximum likelihood, Euclidean Minimum
Distance (EMD)
Spectral Angle Mapper (SAM)
85% up to 92%
[25]Turkey2 crop 3 stages for each and 6 featuresRapid Eye1Support Vector Machine85.63%
[26]12 test sites (4 Europe, 4 Africa, 2 America 2 Asia)4-6 when detailedSPOT 4 and Landsat-8Not detailedRandom forest80%
[27]Peru8Landsat 7 ETM+53Random forest81%
[5]Turkey5SPOT 51Random forest85.89%
[28]USA9ASTER2Neural networks and support vector machines88%
[29]Brazil5Landsat-82Random Forest80%
[30]Germany8Landsat-7 and -8, Sentinel-2A and RapidEye. 36 in 2015
47 in 2016
Fuzzy c-means clustering77.19% up to 89.49%
[31]Spain15Landsat-8 and Sentinel 28761Ensemble Bagged Tree87% up to 92%
[6]USA9ASTER2 Logistic regression (LR) 86%
Table 2. Crop selected for the case study and number of visited plots for each crop.
Table 2. Crop selected for the case study and number of visited plots for each crop.
CropsNumber of Visited Plots
Cereals
Barley, C1431
Maize, C2246
Wheat, C3395
Industrial crops
Poppy, C7125
Sunflower, C841
Deciduous tress
Almond tree, C10103
Vineyard, C11130
Horticultural crops
Onion, C4124
Purple garlic, C595
White garlic, C6100
Perennials (rangegrass)
Alfalfa, C9144
Legumes
Peas, C1298
Table 3. Percentage of cloud cover for Sentinel-2A and Landsat-8 images on each sampling day through the growing season.
Table 3. Percentage of cloud cover for Sentinel-2A and Landsat-8 images on each sampling day through the growing season.
Sentinel-2A *Landsat-8 Sentinel-2A *Landsat-8
DateR051R094Path/Row: 199/33Path/Row: 200/33DateR051R094Path/Row: 199/33Path/Row: 200/33
5-Mar3.78 1.38 18-Jul 0.04
12-Mar 6.37 0.4127-Jul 2.58
25-Mar27.15 30-Jul0.00
1-Apr. 3.86 2-Aug 0.01
6-Apr 3-Aug 0.56
14-Apr.0.68 9-Aug0.00
24-Apr.0.00 12-Aug0.01 9.02
1-May 0.00 19-Aug0.00 0.05
24-May31.97 7.21 1-Sep 4.29
31-May 6.474-Sep 0.04
9-Jun 0.80 11-Sep 32.31
13-Jun 0.01 13-Sep 12.15
20-Jun1.84 21-Sep 0.03
25-Jun 2.44 1-Oct 3.12
2-Jul 0.076-Oct 6.86
3-Jul 0.62 8-Oct36.88
10-Jul0.02 11-Oct 0.01
11-Jul 0.67 15-Oct 0.26
* The required tiles to cover the case study are T30SWJ and T30SXJ.
Table 4. Main statistics of the relationship between the NDVI values obtained with Landsat-8 and Sentinel-2A.
Table 4. Main statistics of the relationship between the NDVI values obtained with Landsat-8 and Sentinel-2A.
Data SizeR2RMSERE
29340.9980.0512.18
29340.9990.0611.70
29420.9710.0623.46
88100.9960.0614.30
Table 5. Classification algorithms evaluated.
Table 5. Classification algorithms evaluated.
Group MethodMain Characteristics
Decision treesM1Complex treeDifferent number of leaves and maximum number of splits (up to 100, 20 and 4 respectively)
M2Medium tree
M3Simple tree
Discriminant analysisM4Linear discriminantBoth are parametric methods, with differences in the determination of the boundaries (linear and quadratic respectively)
M5Quadratic discriminant
Support Vector MachinesM6Linear SVMLinear kernel
M7Quadratic SVMQuadratic kernel
M8Cubic SVMCubic kernel
M9Fine Gaussian SVMGaussian kernel with fine kernel scale (n0.5/4)
M10Medium Gaussian SVMGaussian kernel with medium kernel scale (n0.5)
M11Coarse Gaussian SVMGaussian kernel with coarse kernel scale (n0.5/4)
Nearest NeighborM12Fine KNNEuclidean distance metric. The number of neighbors is set to 1
M13Medium KNNEuclidean distance metric. The number of neighbors is set to 10
M14Coarse KNNEuclidean distance metric. The number of neighbors is set to 100
M15Cosine KNNCosine distance metric. The number of neighbors is set to 10.
M16Cubic KNNCubic distance metric. The number of neighbors is set to 10.
M17Weighted KNNDistance weight. The number of neighbors is set to 10
Ensemble classifiersM18Boosted TreesAdaBoost ensemble method with decision trees
M19Bagged TreesEnsemble method with decision trees
M20Subspace DiscriminantSubspace, with discriminant learners
M21Subspace KNNSubspace ensemble method, with nearest neighbor learners
M22RUSBoost TreesRUSBoost ensemble method, with decision tree learners
Table 6. F1 score statistic of the different classification methods with the available information.
Table 6. F1 score statistic of the different classification methods with the available information.
Decision TreesDiscriminant AnalysisSupport Vector Machine
M1M2M3M4M5M6M7M8M9M10M11
L80.740.690.550.690.770.800.850.850.780.830.71
S20.770.750.580.730.780.830.860.860.780.840.79
L8 and S20.760.740.580.760.660.840.880.870.770.850.79
L8 and S2 corrected0.760.740.580.760.660.840.880.880.770.850.79
Nearest Neighbor ClassifiersEnsemble Classifiers
M12M13M14M15M16M17M18M19M20M21M22
L80.850.790.670.790.760.850.740.850.690.860.71
S20.870.830.720.820.830.870.770.830.730.870.72
L8 and S20.880.820.680.810.810.860.780.860.740.890.74
L8 and S2 corrected0.880.820.680.810.810.860.780.850.740.890.72
Table 7. Confusion matrix of the subspace ensemble method with nearest neighbor learners (M21) for L8 and Sentinel-2A information.
Table 7. Confusion matrix of the subspace ensemble method with nearest neighbor learners (M21) for L8 and Sentinel-2A information.
C1C2C3C4C5C6C7C8C9C10C11C12UA
C110701300400000682.3
C2171010001000095.9
C314010301010000086.6
C4000352000001092.1
C5000027100000193.1
C6000012800100093.3
C70000003800000100.0
C800040009000069.2
C9111000004100093.2
C10000000000301096.8
C11100100000235089.7
C12702000000002170.0
PA81.798.686.685.487.184.897.490.097.693.894.675.0
Table 8. Confusion matrix of the nearest neighbor classifier with the fine KNN method (M12).
Table 8. Confusion matrix of the nearest neighbor classifier with the fine KNN method (M12).
C1C2C3C4C5C6C7C8C9C10C11C12UA
C110801600200000482.3
C2171010001000095.9
C312010401020000086.6
C4100341001001092.1
C5100025100000293.1
C6010002800100093.3
C70000003800000100.0
C800050008000069.2
C9011001004100093.2
C10100000000282096.8
C11101100000234089.7
C12602001000102070.0
PA81.798.686.685.487.184.897.490.097.693.894.675.0

Share and Cite

MDPI and ACS Style

Chakhar, A.; Ortega-Terol, D.; Hernández-López, D.; Ballesteros, R.; Ortega, J.F.; Moreno, M.A. Assessing the Accuracy of Multiple Classification Algorithms for Crop Classification Using Landsat-8 and Sentinel-2 Data. Remote Sens. 2020, 12, 1735. https://doi.org/10.3390/rs12111735

AMA Style

Chakhar A, Ortega-Terol D, Hernández-López D, Ballesteros R, Ortega JF, Moreno MA. Assessing the Accuracy of Multiple Classification Algorithms for Crop Classification Using Landsat-8 and Sentinel-2 Data. Remote Sensing. 2020; 12(11):1735. https://doi.org/10.3390/rs12111735

Chicago/Turabian Style

Chakhar, Amal, Damián Ortega-Terol, David Hernández-López, Rocío Ballesteros, José F. Ortega, and Miguel A. Moreno. 2020. "Assessing the Accuracy of Multiple Classification Algorithms for Crop Classification Using Landsat-8 and Sentinel-2 Data" Remote Sensing 12, no. 11: 1735. https://doi.org/10.3390/rs12111735

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop