https://orcid.org/0000-0002-3583-9434
Figures
Abstract
Decision making is constantly present in agriculture. Choosing the wrong variety carries the risk that the investment in terms of sowing does not pay off at all. Therefore, it is necessary to choose the variety that gives the best results. In order to achieve this, it is necessary to apply multi-criteria decision-making of available varieties, which is, in this paper, done on the example of hybrid varieties of rapeseed that were created by selection at the Institute of Field and Vegetable Crops in Novi Sad. By applying fuzzy logic, a novel integrated Multi-Criteria Decision-Making (MCDM) model is developed and rapeseed varieties were evaluated. For determining four main and 20 subcriteria, fuzzy PIPRECIA (PIvot Pairwise RElative Criteria Importance Assessment) method has been applied based on fuzzy Bonferroni operator, while for ranking alternatives fuzzy MABAC (Multi-Attributive Border Approximation area Comparison) method has been used. The results obtained using the novel integrated fuzzy MCDM model showed that the variety A2 – Zorica has the best results, followed by A1 - NS Ras, while the worst results were seen by the variety A5 - Zlatna. These results were confirmed using other five fuzzy MCDM methods. Sensitivity analysis—changing criteria weights showed the worst results in the variety A6 - Jovana, which took last place in the application of 18 scenarios. The presented model and the results of this research will help farmers to solve this decision problem.
Citation: Nedeljković M, Puška A, Doljanica S, Virijević Jovanović S, Brzaković P, Stević Ž, et al. (2021) Evaluation of rapeseed varieties using novel integrated fuzzy PIPRECIA – Fuzzy MABAC model. PLoS ONE 16(2): e0246857. https://doi.org/10.1371/journal.pone.0246857
Editor: Fausto Cavallaro, Universita degli Studi del Molise, ITALY
Received: October 20, 2020; Accepted: January 27, 2021; Published: February 25, 2021
Copyright: © 2021 Nedeljković et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper.
Funding: We acknowledge support by the German Research Foundation and the Open Access Publication Fund of TU Berlin.
Competing interests: The authors have declared that no competing interests exist.
1. Introduction
Rapeseed is one of the main sources of vegetable oil in the world. In addition to the relatively high and stable price, which is probably a decisive factor in the decision of producers to sow rapeseed on their fields [1], rapeseed has a number of characteristics that enhance its production [2]. It is used to produce the oil of excellent quality for human consumption and the processing industry [3]. Furthermore, it is also used to obtain protein components for domestic animal nutrition [4]. Rapeseed is also recommended in organic agriculture and as a component in joint crops [5, 6].
According to Food and Agriculture Organization (FAO) data, the area of rapeseed reached 39,579,575 ha, and its production 75,001,457 tons in the world in 2018. The production of this oilseed in the European Union (EU) (28) reached the value of 19,974,567 tons in the same year, which immediately put it behind its largest producer, which is Canada (20,342,600 tons). When it comes to the production of this oilseed in Bosnia and Herzegovina, it can be seen that it is very modest and, according to the same source, it amounted to 26,775 tons in 2018, which is only 0.13% of production in the EU (28) or 0.03% in the world.
Considering the growing importance of this industrial plant and the growing production potential it has, many agricultural farms and agro-industrial complexes in Bosnia and Herzegovina should make decision to produce it. The main role in the decision to sow this oilseed is played by the selection and evaluation of the current variety (hybrid) that is available on the market. Rapeseed hybrid selection is necessary to improve production. Constant improvement of existing hybrids and introduction of new ones is needed. In order to achieve this, it is essential to choose the best hybrid and improve it. However, the main problem is the existence of a large number of hybrids. Reducing the number of them is achieved by comparing the existing hybrids. Each of the hybrids should be evaluated and compared to the existing ones in order to determine which of them has the best characteristics.
The aim of this paper is to develop a novel integrated fuzzy MCDM model of support for deciding on potential sowing with some of the analyzed rapeseed hybrids. The realization of this aim will result in the selection of the best hybrid that will serve to further improvement of the rapeseed production. It is necessary to choose the hybrid that shows the best results since it will serve as a basis for creating new rapeseed hybrids. For the purposes of the research, the selection and analysis refer to six varieties of rapeseed hybrids created by selection at the Institute of Field and Vegetable Crops in Novi Sad. The fuzzy MCDM methodology is used to decide on several alternatives, and the evaluation is performed according to criteria anonymously and differently set by several experts in the field. The contribution of this paper is in the following:
- Forming models that will, through the selection of the best hybrid, serve for further improvement of rapeseed hybrids;
- Application of a new methodology based on fuzzy logic;
- Introduction of new analysis for confirming research results.
Fuzzy logic allows the introduction of a mean value defined between the traditional attitudes of yes/no, true/false, black/white, and so on [7]. The use of the fuzzy approach is based on linguistic values provided by the experts. The concept of linguistic values is useful in dealing with situations that are too complex or not well defined to be valued in quantitative terms. Linguistic values are closer to human thinking and are easier to use when evaluating quantitative indicators.
In addition to the introduction, this paper is divided into six other sections. In the second section, a review of the literature on previous research in agriculture using MCDM methods was performed. This was followed by an explanation of the basics of fuzzy logic, fuzzy Bonferroni mean (BM) operator, fuzzy PIPRECIA and the MABAC method presented in the third section. In the fourth section of this paper, the case study was presented. The fifth section was in charge of the results of this research. In the sixth section, sensitivity analysis was applied to confirm the obtained research results. The seventh section was in charge of giving the most important conclusions of this research.
2. Literature review
Decision-making in agriculture is an unavoidable phase in business. It is a complex business since decisions due to the impossibility of quantification are often made on the basis of qualitative data or, even more often, in combination with existing quantitative data [8, 9]. The problem becomes more complicated with the fact that unforeseen climatic conditions and numerous risks, caused by political, economic and social phenomena in the society, are increasingly present. In addition, different varieties and types of fruit and vegetables are available to farmers, and they need to decide what to plant and in what way. In order to make a decision in agriculture, it is necessary to consider certain alternatives that are available with certain criteria. Due to the existence of several criteria that need to be considered when making a decision, multi-criteria decision analysis (MCDA) methods are used.
Deepa et al. [10] used the methods VIKOR (Multicriteria Optimization and Compromise Solution), TOPSIS (Technique for Order Preference by Similarity to an Ideal Solution), entropy, and standard deviation (VTOPES) for ranking sustainable farms for sugar cane production. Dooley et al. [11] discussed MCDA methods and their application in agriculture on the example of three case studies and provided guidelines on how these methods can be used in agriculture. Crnčan et al. [12] used the AHP (Analytic Hierarchy Process) method in the selection of egg production systems based on the assessment of the respondents. Latinopoulos [13] used the basics of the MCDA to develop a MOP (Mathematical Optimization Programming) model that served to efficiently allocate water and land resources in rural Greece. Rozman et al. [14] used the DEX method (Decision EXpert) to evaluate dairy farms in Bosnia and Herzegovina. Bogdanović and Hadžić [15] performed a strategic analysis of investments in agriculture using the AHP method and found that perennial plantations are a better long-term investment strategy than conventional crop production.
Pozderec et al. [16] used the DEX and AHP methods to integrate the developed specific technological-economic simulation models for vegetable production into a greenhouse in integrated and organic production. Bartzas and Komnitsas [17] combined life cycle analysis, environmental risk assessment together with the AHP method to select the most sustainable agricultural management practices at the regional level. Yazdani et al. [18] used the extended Step-wise Weight Assessment Ratio Analysis method combining gas and multicriteria analysis to manage the circular economy in agriculture. Nikoloski et al. [19] used the DEX method to create a model of decision support on farm relocation in horticulture based on criteria regarding region and specific farm. Elleuch et al. [20] used Fuzzy Multi-Criteria Decision-Making (FMCDM) methods and MOP for the water distribution problem.
Based on this brief review of the literature, it could be seen that different MCDA methods were used and different problems in agriculture were solved. Jayakumar and Hari Ganesh [21] used the fuzzy multicriteria group decision making (FMCGDM) model for rice variety selection. Draginčić et al. [22] used the Simple Additive Weighting (SAW) method, the AHP and Cardinal consensus model using the Group multi-criteria decision making (GMCDM) when selecting a grape variety. Rozman et al. [23] used the DEX method to select the plum variety that gives the best results in integrated fruit production Maksimović et al. [24] used the DEX method to select the apple variety that gives the best results in North-western Bosnia and Herzegovina. Also, Maksimović et al. [25] used the DEX method to select the fruit species that give the best results in integral fruit production. Milovanović and Stojanović [26] used the AHP method to select cherry varieties. Rozman et al. [27] used the AHP method when evaluating new apple cultivars. Paunović et al. [28] used the Fuzzy Interference System (FIS) to evaluate cherry varieties for orchard raising purposes. Based on a brief review of the literature, it can be concluded that MCDA methods can be used to evaluate different varieties for sowing purposes.
3. Methods
Fuzzy logic is used to model inaccurate information derived from human thinking [29]. In order to make a decision, human subjectivity should be taken into account, and not only the objective probabilities of the occurrence of an event, because it is not possible to have complete information. When making decisions to deal with very complex problems, one does not have to use rigidity to specify the description and thinking about phenomena as much as possible, but one can go in the opposite direction and allow the descriptions to be imprecise in the spirit of natural language [30].
3.1. Operations on fuzzy numbers
A fuzzy number on R to be a triangular fuzzy number (TFN) if its membership function
: R→[0,1] is equal to following Eq (1):
(1)
From Eq (1), l and u mean the lower and upper bounds of the fuzzy number , and m is the modal value for
. The TFN can be denoted by
.
The operational laws of TFN and
are displayed as following equations [31–33].
3.2. Fuzzy Bonferroni mean (BM) operator
Surveys were conducted anonymously to collect the data necessary for this research from five experts in the field. Since five experts participate in the research, the values of the interval fuzzy vector of weight coefficients are aggregated using a fuzzy Bonferroni aggregator [34], Eq (7). The Bonferroni mean (BM) operator [35] was used to aggregate the vector of weight coefficients because it allows the representation of the interrelationships between the aggregation elements [36].
(7)
where e represents the number of experts participating in the research, while p, q ≥ 0 are set of non-negative numbers.
3.3. Fuzzy PIPRECIA method
Steps of this method are the following [9, 37, 38]:
Step 1. Forming set of criteria and sorting the criteria according to marks from the first to the last, and this means that they need to be sorted unclassified.
Step 2. Each decision-maker individually evaluates pre-sorted criteria by starting from the second criterion.
denotes the assessment of criteria by a decision-maker r.
Step 3. Determining the coefficient
(9)
Step 4. Determining the fuzzy weight
(10)
Step 5. Determining the relative weight of the criterion
(11)
In the following steps, the inverse methodology of fuzzy PIPRECIA method requires to be applied.
Step 6. Performing the assessment but this time starting from a penultimate criterion.
Step 7. Determining the coefficient
(13)
Step 8. Determining the fuzzy weight
(14)
Step 9. Determining the relative weight of the criterion
(15)
Step 10. In order to determine the final weights of criteria, it is necessary to perform the defuzzification of the fuzzy values and
first.
Step 11. Checking the results obtained by applying Spearman and Pearson correlation coefficients.
1.1. Fuzzy MABAC method
Multi-Attributive Border Approximation area Comparison (MABAC) developed by Pamučar and Ćirović [39] is one of the recent approaches to multi-attribute decision making (MCDM/MADM) problems. The basic assumption of the MABAC method is reflected in the definition of the distance of the alternative from the boundary approximate domain. The boundary approximate area represents the average value for all alternatives. If the alternative is above that value, its value will be positive and vice versa.
The application of the fuzzy MABAC method is done by using 7 steps [40].
Step 1. Forming the initial decision matrix. In the first step the evaluation of m alternatives by n criteria is performed.
Step 2. Normalization of the initial matrix elements. The elements of the normalized matrix are obtained by using the expressions:
For benefit-type criteria:
(17)
For cost-type criteria:
(18)
where l is the first fuzzy number, m is the second fuzzy number and u is the third fuzzy number.
Step 3. Calculation of the weighted matrix (V) elements [41].
(19)
where wi represents the weighted coefficients of the criterion.
Step 4. Determination of the approximate border area matrix (G).
(20)
where m represents total number of alternatives.
Step 5. Calculation of the matrix elements of alternatives distance from the border approximate area. The distance of the alternatives from the border approximate area () is defined as the difference between the weighted matrix elements (V) and the values of the border approximate areas (G).
Now, boarder approximation area value for each criteria function serves as reference point/benchmark value for criteria-wise performance of an alternative Ai. Each individual candidate will belong to the three different areas namely the border approximation area (G), upper approximation area (G+), and lower approximation area (G−). The ideal alternative () can be found in the upper approximation area (G+) whereas the lower approximation area (G−) contains the anti-ideal alternative (
) [42].
In order for the alternative Ai to be chosen as the best from the set, it is necessary for it to belong, by as many as possible criteria, to the upper approximate area () [43]. The higher value
indicates that the alternative is closer to the ideal alternative, while the lower value q
indicates that the alternative is closer to the anti-ideal alternative [44].
Step 6. Ranking of alternatives. The calculation of the values of the criteria functions by alternatives is obtained as the sum of the distance of alternatives from the border approximate areas (). By summing up the matrix
elements per rows, the final values of the criteria function of alternatives are obtained
(23)
Step 7. Final ranking of alternatives. By defuzzification of the obtained values , the final rank of alternatives is obtained.
The best alternative is the one that has the greatest value [45].
4. Case study
From the literature review it could be seen that there are different methods of MCDA that can be used in different problems in agriculture. In order to pursue the aim of the research in this paper, the evaluation of rapeseed varieties, originated at the Institute of Field and Vegetable Crops in Novi Sad, will be performed. Each of these varieties has its advantages and disadvantages, so this paper will develop a decision model that will serve as the basis for a detailed analysis of these varieties. By analysing rapeseed hybrids produced at the Institute of Field and Vegetable Crops in Novi Sad, the best hybrid that will serve as the basis for the creation of new hybrids, will be selected. Methods developed on the principles of fuzzy logic will be used. The following steps were used to collect data for the purposes of this study.
Step 1 Selection of criteria according to which these varieties of rapeseed will be evaluated. In this step, a review of previous research was performed first and certain criteria were selected. Anonymous surveys were organized to collect data from the experts who performed the evaluation. They selected several criteria that were grouped into four groups according to which the evaluation of rapeseed varieties would be performed. Based on these criteria, a decision-making model was created (Fig 1). The decision model has four main criteria: C1 stem or fruit, C2 resistance of the plant or variety, C3 transport, storage and processing of varieties, C4 economic criterion. All these criteria are further divided into five sub-criteria. The reason for this was that all the criteria would have the equal number of sub-criteria and so that none of them would stand out. Each of the criteria and sub-criteria will be evaluated by experts using linguistic values from Fuzzy PIPRECIA method.
Step 2 Selecting alternatives to be evaluated. For the purposes of this research, the experts selected six different hybrid varieties of rapeseed: A1—NS Ras, A2—Zorica, A3—Anna, A4—Jasna, A5—Zlatna and A6—Jovana. Selected rapeseed varieties will be evaluated using linguistic values ranging from Very bad (VB) to Very good (VG) values. As with the evaluation of criteria, seven linguistic assessments are available for the evaluation of alternatives. By using linguistic assessments, decision-making approaches human thinking and is much easier.
Step 3 Data collection and ranking of alternatives. After the experts have evaluated the criteria and alternatives, the assessments are collected and the initial decision matrix is formed, which is the basis for the implementation of the MABAC method. Prior to that, all criteria weights are calculated using fuzzy PIPRECIA method based on fuzzy BM operator. The steps of the fuzzy MABAC method are then applied and the ranking orders of the alternative are determined. To confirm these results, a sensitivity analysis will be performed where other results will be checked using other fuzzy methods. Furthermore, it will examine how sensitive the alternatives are to changes in the weights of the sub-criteria.
5. Research results
Before ranking hybrid varieties of rapeseed, it is necessary to calculate the weights of the criteria. The experts first evaluated the main criteria and then the sub-criteria for the individual main criteria. In Table 1 the evaluation of main criteria by all experts and their aggregation using fuzzy operator is presented.
Based on the obtained weights for the main criteria, it can be seen that the most important criteria for experts are C4 economic criterion and C2 resistance of the plant / variety. The least important criterion in their opinion is the C1 stem / fruit criterion.
Multiplying the weights of the main criteria with the weights of the sub-criteria, the final values of the weights of the sub-criteria are obtained (Table 2).
After determining the weights of the criteria, the experts evaluated the hybrid varieties of sugar beet with assessments in the form of linguistic values (Table 3).
These linguistic values are transformed into fuzzy numbers using the affiliation function. The mean value of the expert opinion is then calculated using the geometric mean. In this way, a common fuzzy decision matrix is formed (Table 4).
After the common decision matrix is formed, the maximum value for the sub-criteria is calculated and this decision matrix is normalized. The next step of the MABAC method is to multiply the normalized decision matrix by the corresponding weights and add them to those weights. In such a way, an extended decision matrix was obtained. An approximate border area matrix is then determined and these values are subtracted from the extended decision matrix. The next step is to add the appropriate values of the fuzzy numbers and phase the fuzzy numbers. The alternatives are then ranked so that the best alternative is the one with the highest Si value (Table 5). In this research, the best ranked hybrid variety of sugar beet is A2—Zorica, followed by the A1—NS Ras, while the worst ranked variety is, according to results, A6—Jovana.
In order to confirm the obtained results using fuzzy MABAC, the application and calculation of rank orders will be performed using other fuzzy methods.
6. Sensitivity analysis and comparison analysis
Sensitivity analysis was three ways. The first way was to examine the results obtained by the fuzzy MABAC method. This was done in such a way that the ranking order obtained by this method was compared with the ranking orders obtained using other fuzzy methods [46]. Another way to conduct sensitivity analysis is to examine how a change in the weight of sub-criteria affects the ranking order of alternatives [47].
The ranking of alternatives was performed using the following methods: fuzzy WASPAS (Weighted Aggregated Sum Product Assessment) [31], fuzzy SAW (Simple Additive Weighting technique) [31], fuzzy MARCOS (Measurement Alternatives and Ranking according to the Compromise Solution) [48], fuzzy ARAS (Additive Ratio Assessment) [31], fuzzy TOPSIS (Technique for Order Performance by Similarity to Ideal Solution) [49]. The results obtained show that there is no difference in the ranking of alternatives (Fig 2), so it can be concluded that the fuzzy MABAC method can be used to determine the ranking of alternatives in this case hybrid varieties of rapeseed, because it gives the same result as other fuzzy methods. As evidenced by this sensitivity analysis in these decision-making problems in agriculture, fuzzy logic and various fuzzy methods can be used.
When conducting a second sensitivity analysis, it is necessary to use different scenarios that represent different weights of the sub-criteria. Since there are 20 sub-criteria in the decision-making model, 21 scenarios were used here, and in 20 scenarios one of the criteria was assigned the weight 0.24 while the other criteria were assigned the weight 0.04, in 21 scenarios all sub-criteria were assigned the same weight:” (Table 6). In this way, in the first 20 scenarios, more weight was given to individual criterion than to other criteria, and the impact of this sub-criterion on the ranking of the alternative was observed. The 21st scenario does not give preference to any sub-criteria, but observes all them equally.
The results obtained by applying different scenarios show that as many as 3 hybrid varieties of rapeseed occupy the first place by applying a certain scenario. Alternative A2 takes first place in the ranking in 12 scenarios, alternative A1 takes first place in 8 scenarios while alternative A3 takes first place once in the application of scenario 20. In this way you can see which scenarios are sensitive to each alternative. Take for example alternative A2, it is sensitive to 8 scenarios because then it takes second or third place. So, we can say that this alternative is particularly sensitive to the application of scenario 14 when it took third place in the ranking. Alternative A2 has a poorer ability to store fruit compared to alternatives A1 and A3. The situation is similar in the other 8 scenarios where alternatives A1 and A3 have a better ranking compared to alternatives A2. Alternative A4 did not show any sensitivity to changes in the weights of the sub-criteria as it ranked fourth in all scenarios used. Alternative A5 ranked fifth in 18 scenarios while Alternative A6 ranked fifth in 3 scenarios. Alternatives A5 and A6 showed the worst indicators with this analysis. In order for any alternative to be better, it must improve the characteristics according to a certain sub-criterion that influenced the lower ranking. To conclude, it is necessary to improve that variety in terms of that sub-criterion (Fig 3).
The Bonferroni aggregator was applied to aggregate expert’s preferences in calculating the values of weight coefficients in the fuzzy PIPRECIA model. When calculating the initial values of the fuzzy weights of the criteria in the Bonferroni function, the parameter values were adopted p = q = 1. Many studies [35, 50, 51] indicate the fact that changes in the values of the parameters p and q can lead to changes in the aggregate values, and thus in the final rang of alternatives. Therefore, the following section presents an analysis of the impact of changes in the parameters p and q on the change in the values of the weight coefficients of the criteria and values in the aggregate decision matrix.
In the next part, the influence of changing the parameters p and q through 300 scenarios is simulated and 300 new vectors of weight coefficients are formed. Thus, through the first 100 scenarios, the change of the parameter value was simulated 0≤p≤100, while value of parameter q staying same, ie. q = 1. In the next 100 scenarios the change of the parameter value was simulated 0≤q≤100, while value of parameter p is unchanged ie. p = 1. Fig 4A and 4B. In the last 100 scenarios changing of both parameters in range 1≤p,q≤100, is simulated (Fig 4C).
After determining the new vectors of weight coefficients, their influence on the change of the values of the criterion functions in the fuzzy MABAC model was analyzed (Fig 5).
By analyzing the results from Fig 5, we notice that for changes in the values of parameters through all 300 do not lead to a change in the rank of alternatives and that the initial rank A2> A1> A3> A4> A5> A6 is confirmed. It has been confirmed that changes in the values of the parameters p, q lead to a change in the values of the criterion functions, however these changes are not large enough to cause changes in the initial rank. Also, the presented results showing that the initial solution is credible and robust despite drastic changes in the input parameters.
7. Conclusion
Each sowing is an investment for farmers. They have to invest certain funds for land preparation, seed procurement and sowing, and the return on that investment is expected after the harvest. Therefore, when choosing a variety, it is necessary to choose the one that will bring them the best results. In order to be able to choose a certain variety at all, it is necessary to consider the characteristics of that variety. The characteristics are considered on the basis of several criteria, and MCDM methods are used to make a decision on the source of the variety.
This paper, on the example of hybrid varieties of rapeseed developed at the Institute of Field and Vegetable Crops in Novi Sad, ranked these varieties using a novel integrated fuzzy MCDM model. Fuzzy logic enables the application of linguistic values or evaluations for assessment as close as possible to human thinking. In order to apply fuzzy logic, it was necessary to create a decision model for this problem. Assistance of five anonymous experts was used to create a decision-making model on the basis of which individual varieties were evaluated. The fuzzy MABAC method was used to rank these alternatives.
The results obtained using the fuzzy MABAC method showed that based on the weights determined by five experts, the hybrid rapeseed variety A2—Zorica has the best results, followed by the A1 NS Ras variety, while the A5—Zlatna variety has the worst results. These results were confirmed using other fuzzy methods. Sensitivity analysis showed that variety A6—Jovana has the worst results, taking the last place in the application of 18 scenarios.
The main limitation of this research is the analysis of 6 hybrids produced at the Institute of Field and Vegetable Crops in Novi Sad. Taking a larger number of hybrids implies the problem of evaluating them. It is of huge importance to give equal conditions to all hybrids and monitor them throughout the year, and only then, based on that monitoring, it is necessary to give an assessment for each hybrid. As this number increases, the process of selecting the best hybrid becomes more complicated.
The conducted research provided good settings on how to solve the problem of decision-making in agriculture regarding the choice of variety for sowing. In addition, the obtained results showed advantages and disadvantages of certain hybrid varieties of rapeseed. These results will help farmers choose the variety of rapeseed that will give them the best results.
In future research, it would be suitable to apply this methodology in order to perform the analysis of other hybrids and thus select the best basis for the development of new types of hybrids. Only on the best hybrids it is possible to create even better samples and thus increase the production of rapeseed by farmers.
References
- 1. Popović R., Jeremić M., & Matkovski B. (2016). Tržište uljarica u Srbiji. Ratarstvo i povrtarstvo, 53(2), 74–80.
- 2. Crnobarac J., Marinković B., Balalić I., Latković D., & Jaćimović G. (2013). The influence of sowing date on yield and quality of confectionary sunflower. Agriculture & Food 1, 56–63.
- 3. Lin L., Allemekinders H., Dansby A., Campbell L., Durance-Tod S., Berger A., et al. (2013). Evidence of health benefits of canola oil. Nutrition Reviews, 71(6), 370–385. pmid:23731447
- 4. Marjanović Jeromela A., Dimitrijević A., Terzić S., Mikić A., Atlagić J., Miladinović D., et al. (2016). Applying Mendelian rules in rapeseed (Brassica napus) breeding. Genetika, 48(3), 1077–1086.
- 5. Antanasović S., Ćupina B., Krstić Đ., Manojlović M., Čabilovski R., Marjanović Jeromela A., et al. (2012). Potential of autumnsown rapeseed (Brassica napus) as a green manure crop. Cruciferae Newsletter, 31, 26–28.
- 6. Mikić A., Antanasović S., Ćupina B., Marjanović Jeromela A., Erić P., & Jakšić S. (2015). Intercropping autumn-sown brassicas with annual legumes for green manure. Cruciferae Newsletter, 34, 11–13.
- 7. Božanić D. I. & Pamučar D. S. (2010). Vrednovanje lokacija za uspostavljanje mosnog mesta prelaska preko vodenih prepreka primenom fuzzy logike. Vojnotehnički glasnik, 58(1), 129–145.
- 8. Blagojevića B., Srđević B. Srđević Z. & Zoranović T. (2017). Grupno odlučivanje pomoću Analitičkog hijerarhijskog procesa. Annals of agronomy, 41(1), 30–39.
- 9. Stević Ž., Stjepanović Ž., Božičković Z., Das D. K., & Stanujkić D. (2018). Assessment of conditions for implementing information technology in a warehouse system: A novel fuzzy piprecia method. Symmetry, 10(11), 586., 2020
- 10. Deepa N., Durai R. V. P M., Srinivasan K., Chang C.-Y., & Bashir A. K. (2019). An Efficient Ensemble VTOPES Multi-Criteria Decision-Making Model for Sustainable Sugarcane Farms. Sustainability, 11(16), 4288.
- 11. Dooley A. E., Smeaton D. C., Sheath G. W., & Ledgard S. F. (2009). Application of multiple criteria decision analysis in the New Zealand agricultural industry. Journal of Multi-Criteria Decision Analysis, 16(1–2), 39–53.
- 12. Crnčan A., Kralik I., Kristić J. & Hadelan L. (2016). Application of Multi-Criteria Decision-Making Methods in Agricultural Production. Krmiva, 58(1), 33–40.
- 13. Latinopoulos D. (2009). Multicriteria decision-making for efficient water and land resources allocation in irrigated agriculture. Environment, Development and Sustainability, 11(2), 329–343.
- 14. Rozman Č., Grgić Z., Maksimović A., Ćejvanović F., Puška A., Šakić B., et al. (2016). Multiple-criteria approach of evaluation of milk farm models in Bosnia and Herzegovina. Mljekarstvo, 66(3), 206–214. pmid:28516419
- 15. Bogdanović S., & Hadžić M. (2019). Strategic multicriteria decision-making process in agriculture. Ekonomika poljoprivrede, 66(1), 89–106.
- 16. Pozderec S., Bavec M., Rozman Č., Vinčec J., & Pažek K. (2015). Multi-Criteria Assessment of Vegetable Production Business Alternatives. Organizacija, 48(3), 203–213.
- 17. Bartzas G., & Komnitsas K. (2020). An integrated multi-criteria analysis for assessing sustainability of agricultural production at regional level. Information Processing in Agriculture. 7(2), 223–232.
- 18.
Yazdani M., Gonzalez E.D.R.S. & Chatterjee P. (2019). A multi-criteria decision-making framework for agriculture supply chain risk management under a circular economy context. Management Decision, article in press. https://doi.org/10.1108/MD-10-2018-1088
- 19. Nikoloski T., Udovč A., Pavlovič M., & Rajkovič U. (2018). Multi-criteria assessment model for farm reorientation. Journal of Decision Systems, 27(sup1), 79–87.
- 20. Elleuch M. A., Anane M., Euchi J., & Frikha A. (2019). Hybrid fuzzy multi-criteria decision making to solve the irrigation water allocation problem in the Tunisian case. Agricultural Systems, 176, 102644.
- 21. Jayakumar S., & Hari Ganesh A. (2012). Fuzzy Multi Criteria Group Decision Making (Mcgdm) Approach for Variety Selection in Rice Farming. Australian Journal of Basic and Applied Sciences, 6(12): 308–318.
- 22. Draginčić J., Korać N., & Blagojević B. (2015). Group multi-criteria decision making (GMCDM) approach for selecting the most suitable table grape variety intended for organic viticulture. Computers and Electronics in Agriculture, 111, 194–202.
- 23. Rozman Č., Maksimović A., Puška A., Grgić Z., Pažek K., Prevolšek B., et al. (2017). The Use of Multi Criteria Models for Decision Support System in Fruit Production. Erwerbs-Obstbau, 59(3), 235–243.
- 24. Maksimović A., Grgić Z., Puška A., Šakić Bobić B., & Ćejvanović F. (2018). Primjena višekriterijskog odlučivanja za izbor optimalne sorte jabuke za sjeverozapadnu regiju BiH. Journal of Central European Agriculture, 19(3), 740–759.
- 25. Maksimović A., Grgić Z., & Ćejvanović F. (2017). Multi-attribute analysis of orchard according to the integrated production concept. Ekonomika poljoprivrede, 64(1), 69–79.
- 26. Milovanović Ž., & Stojanović M. (2016). Izbor sorti višnje za sadnju primenom AHP metodologije. Agroekonomika, 45(72), 11–19.
- 27. Rozman Č., Hühner M., Kolenko M., Tojnko S., Unuk T., & Pažek K. (2015). Apple Variety Assessment with Analytical Hierarchy Process. Erwerbs-Obstbau, 57(2), 97–104.
- 28. Paunović M., Milutinović O., & Puzić G. (2018). Personal subjectivity impact reduction in choice of sour cherry varieties for orchard establishment using fuzzy system. Economics of Agriculture, 65(2), 545–554.
- 29. Ashrafzadeh M., Rafiei F. M., Isfahani N. M., & Zare Z. (2012). Application of fuzzy TOPSIS method for the selection of Warehouse Location: A Case Study. Interdisciplinary journal of contemporary research in business, 3(9), 655–671.
- 30. Zadeh L. A. (1965). Fuzzy sets. Information and Control, 8(3), 338–353.
- 31. Petrović G., Mihajlović J., Ćojbašić Ž., Madić M., & Marinković D. (2019). Comparison of three fuzzy MCDM methods for solving the supplier selection problem. Facta Universitatis, Series: Mechanical Engineering, 17(3), 455–469.
- 32. Vesković S., Milinković S., Abramović B., & Ljubaj I. (2020). Determining criteria significance in selecting reach stackers by applying the fuzzy PIPRECIA method. Operational Research in Engineering Sciences: Theory and Applications, 3(1), 72–88.
- 33. Durmić E., Stević Ž., Chatterjee P., Vasiljević M., & Tomašević M. (2020). Sustainable supplier selection using combined FUCOM–Rough SAW model. Reports in Mechanical Engineering, 1(1), 34–43.
- 34. Pamučar D., Deveci M., Canitezd F., Bozanic D. (2020). A Fuzzy Full Consistency Method-Dombi-Bonferroni Model for Priorititizing Transportation Demand Management Measures. Applied Soft Computing, 87
- 35. Bonferroni C. (1950). Sulle medie multiple di potenze. Bollettino Matematica Italiana, 5, 267–270.
- 36. Pamučar D. (2020). Normalized weighted Geometric Dombi Bonferoni Mean Operator with interval grey numbers: Application in multicriteria decision making. Reports in Mechanical Engineering, 1(1), 44–52.
- 37.
Đalić I., Ateljević J., Stević Ž., & Terzić S. (2020). An integrated SWOT–fuzzy PIPRECIA model for analysis of competitiveness in order to improve logistics performances. Facta Universitatis, Series: Mechanical Engineering.
- 38. Memiş S., Demir E., Karamaşa Ç., & Korucuk S. (2020). Prioritization of road transportation risks: An application in Giresun province. Operational Research in Engineering Sciences: Theory and Applications, 3(2), 111–126.
- 39. Pamučar D., & Ćirović G. (2015). The selection of transport and handling resources in logistics centers using Multi-Attributive Border Approximation area Comparison (MABAC). Expert Systems with Applications, 42(6), 3016–3028.
- 40. Bozanic D., Tešić D., & Milićević J. (2018). A hybrid fuzzy AHP-MABAC model: Application in the Serbian Army–The selection of the location for deep wading as a technique of crossing the river by tanks. Decision Making: Applications in Management and Engineering, 1(1), 143–164.
- 41. Bozanić D., Tešić D., & Kočić J. (2019). Multi-criteria FUCOM–Fuzzy MABAC model for the selection of location for construction of single-span bailey bridge. Decision Making: Applications in Management and Engineering, 2(1), 132–146,
- 42. Roy J., Chatterjee K., Bandhopadhyay A., Kar S. (2016). Evaluation and selection of Medical Tourism sites: a rough AHP based MABAC approach, arXiv:
- 43. Liu R., Hou L.-X., Liu H.-C., & Lin W. (2020). Occupational health and safety risk assessment using an integrated SWARA-MABAC model under bipolar fuzzy environment. Computational and Applied Mathematics, 39(4), Article number: 276 (2020)
- 44. Gong J‐W., Li Q., Yin L., Liu H‐C. (2019). Undergraduate teaching audit and evaluation using an extended MABAC method under q‐rung orthopair fuzzy environment. International Journal of Intelligent Systems, 35(12), 1912–1933.
- 45. Liu H.-C., You J.-X., & Duan C.-Y. (2017). An integrated approach for failure mode and effect analysis under interval-valued intuitionistic fuzzy environment. International Journal of Production, 207, 163–172.
- 46. Pamučar D., & Božanić D. (2019). Selection of a location for the development of multimodal logistics center: Application of single-valued neutrosophic MABAC model. Operational Research in Engineering Sciences: Theory and Applications, 2(2), 55–71.
- 47. Puška A., Stojanović I., Maksimović A., & Osmanović N. (2020). Evaluation software of project management used measurement of alternatives and ranking according to compromise solution (MARCOS) method. Operational Research in Engineering Sciences: Theory and Applications, 3(1), 89–102.
- 48. Stanković M., Stević Ž., Das D. K., Subotić M., & Pamučar D. (2020). A new fuzzy MARCOS method for road traffic risk analysis. Mathematics, 8(3), 457.
- 49. Chatterjee P., & Stević Ž. (2019). A two-phase fuzzy AHP-fuzzy TOPSIS model for supplier evaluation in manufacturing environment. Operational Research in Engineering Sciences: Theory and Applications, 2(1), 72–90.
- 50. Pamučar D., & Janković A. (2020). The application of the hybrid interval rough weighted Power-Heronian operator in multi-criteria decision making. Operational Research in Engineering Sciences: Theory and Applications, 3(2), 54–73.
- 51. Sinani F., Erceg Z., & Vasiljević M. (2020). An evaluation of a third-party logistics provider: The application of the rough Dombi-Hamy mean operator. Decision Making: Applications in Management and Engineering, 3(1), 92–107.