DETERMINING THE EFFECTIVENESS OF COUNTRIES IN COMBATING PANDEMICS: COVID-19 CASE

. The aim of this study is to develop a multidimensional integrated efficiency analysis (MDIEA) model to be implemented when a pandemic breaks out. The first phase of the model involved the determination of input and output criteria that might affect the course of combat against the pandemic. Two methods were employed in the second phase: the CRITIC method, which uses objective values to determine criteria weights; and, the Fuzzy SWARA method, which is based on subjective values and decision-makers opinions due to the assumption that the criteria might not be determined precisely and completely. The subjective and objective criteria weights obtained from these two methods were combined using the SOWIA method so that integrated criteria weights could be determined. In the third phase, three separate efficiency rankings were obtained using EATWOS, OCRA, and FDEA methods depending on the criteria weights. In the fourth phase, these efficiency rankings were combined by employing the TPOP method to obtain one single precise ranking. The previous studies mostly analyzed the efficiency of countries by using one single efficiency method, where, the model proposed in this study determines countries’ efficiency by employing three different efficiency analysis methods together; it makes use of different points of view and different calculation procedures of different methods. As another aspect, rural population ratio, diabetes prevalence rate, proportion of the population over 65 years old, number of intensive care beds and number of vaccine doses are study-specific criteria, unlike other studies. The present study is expected to contribute to the literature since it is the first study that proposes a model to evaluate the performances of countries during pandemics. The proposed model was implemented to assess OECD countries’ efficiency in their fight against COVID-19, which was a prevailing crisis worldwide. When all the obtained results are considered, it is difficult to claim that one single criterion is effective in combating the pandemic. For an effective fight against the pandemic, it is of great importance to keep the rate of infection and therefore the number of infected people under control. It is also possible for countries to change their policies or adopt stricter policies to contain the spread of the virus.

one of the deadliest pandemics in world history. An estimated 500 million people were infected with the virus worldwide and almost one-fifth of the infected people lost their lives within 18 months. The Plague, which broke out in the southeast of Asia in the 1300s, killed between 75 and 200 million people while HIV, which was first detected in the early 1900s and still has its effects today, was responsible for the death of almost 36 million people. Nearly, 24 million people died because of Asian Flu, which broke out in China in 1957 [2].
First detected on December 1st, 2019 in Wuhan, the capital of the Hubei region of China, and declared as a global pandemic by World Health Organization (WHO) on March 11th, 2020, COVID-19 has immeasurably affected many areas worldwide ranging from the economy to social life and from tourism activities to commercial activities and triggered the emergence of a new era called "new world order". As of December 30th, 2021, more than 5 million people died of COVID-19 and the total number of people infected with the virus reached 284 million worldwide [3]. Since the COVID-19 virus reproduction rate was considerably higher than its previous variants (SARS, MERS), the capacity and quality of health services have become more vital when the attempts to control the spread of the virus are considered. If the number of infected people exceeds the capacity of health facilities, it becomes incredibly challenging to control the pandemic and the death toll increases accordingly [4]. Therefore, countries must have adequate health resources and display effective performance while providing health services for their citizens.
Health services have been recognized as a basic human right worldwide for many centuries. In addition, the provision of quality health services for their citizens has become an area of competition for countries. High standards in health services and healthy citizens are among the significant factors affecting the development of a country. There are qualitative and quantitative health indicators that might be taken into consideration while evaluating to what extent countries can plan and coordinate their health services effectively and efficiently [5]. Such health indicators provide information about the current situation and allow authorities to compare their country with other countries by observing how the nature and variety of problems in healthcare systems change over time [6]. Although the provision of healthcare services might differ due to economic, historical, and cultural differences among countries, the primary common goal of all healthcare services is to provide efficient health services for all citizens at minimum cost [7]. Therefore, some studies were carried out to examine the efficiency and effectiveness of healthcare systems during the presence or absence of pandemics regarding various health indicators.
The present study proposes a new Multidimensional Integrated Efficiency Analysis (MDIEA) model that can be used to determine the efficiency of countries in combating the pandemic and to compare these countries according to their health indicators. The developed model can measure the efficiency of countries during a pandemic. The first stage of the model was to determine the input and output criteria that could affect the course of the fight against the pandemic. In the second stage, two methods were used: the Criteria Importance Through Intercritera Correlation (CRITIC) method, which uses objective values to determine criterion weights [8]; and the Fuzzy Step-wise Weight Assessment Ratio Analysis (Fuzzy SWARA) method, which is based on subjective values and the opinions of decision makers due to the assumption that the criteria cannot be determined fully and precisely [9]. The subjective and objective criterion weights obtained from these two methods were combined using the Subjective and Objective Weight Integrated Approach (SOWIA) method to determine the integrated criterion weights [10]. In the third stage, three different efficiency rankings were obtained by using Efficiency Analysis Technique with Output Satisficing (EATWOS), Operational Competitiveness Rating (OCRA), and Fuzzy Data Envelopment Analysis (FDEA) methods depending on the criterion weights [11][12][13]. Unlike other efficiency measurement methods, the EATWOS method allows decision makers to find satisfactory solutions for decision-making units (DMUs) -i.e., alternatives -in addition to measuring maximum efficiency for input and output criteria [14]. OCRA is a user-friendly and practical method that can efficiently analyze sensitivities in input and output criteria when there is more than one input or output criterion. However, the input and output criteria to be used in the present study are dynamic data that might change over time. Moreover, it is not possible to collect the data for all the criteria for the same periods in a complete and up-to-date way. Analyzing the data after defining it for certain intervals will prevent data loss and allow researchers to obtain more accurate results. Therefore, it will be more convenient to use FDEA here. In the last phase, the efficiency rankings obtained from all three methods were combined using the Technique of Precise Order Preference (TPOP) method to have one single precise ranking [15]. In other words, this integrated ranking reflects the results of different rankings obtained by following different calculation procedures. Figure 1 below shows how this study and the MDIEA model proposed within the scope of the study might contribute to the literature: The proposed model has been employed in order to determine to what extent Organization for Economic Cooperation and Development (OECD) countries are efficient in their combat against the pandemic and compare these countries according to certain health indicators. OECD is an international consulting organization that consists of economy experts [16]. OECD countries were included in the present study because it is convenient to access the data about these countries and the results can be compared to those of similar studies in the literature. Some improvement suggestions were also made within the scope of the study for inefficient countries.
The next section presents the literature review. The proposed model and its implementation process are explained in detail in the third section, which is followed by the results and the discussions in the final section.

Literature review
As a result of the literature review, it has been determined that there are a number of studies focusing on various input and output criteria related to the COVID-19 pandemic. Among the input or output criteria or assessment criteria often used in these studies are the number of COVID-19 cases, death toll, and death rate, the amount of money allocated for health services, expenditures for health services per capita, number of doctors, number of nurses, number of hospitals beds, number of tests done and number of people recovered from COVID-19. There were some other different criteria focused on in the related studies such as the number of respirators, number of health employees, total public funding, number of quarantine centers, number of quarantined people, population, elderly population, literacy rate, population density, and physical immobility.
In studies on COVID-19, techniques such as mathematical modeling, artificial intelligence, machine learning, and simulation are often used to determine the rate of spread of the pandemic, to diagnose the disease, to assess the effect of the measures taken in preventing the spread, and to reveal the effectiveness of countries in the fight against the pandemic [17][18][19][20][21][22]. However, there are COVID-19 studies in the literature using Multi-Criteria Decision Making (MCDM) methods. For instance, Yigit [23] determined OECD countries' performance in their combat against COVID-19 by using The Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) method. The study results showed that Slovakia, Latvia, Korea, New Zealand and Australia were the most efficient countries in the ranking. Maqbool and Khan [24] employed the Decision Making Trial and Evaluation Laboratory (DEMATEL) method in order to analyze the obstacles preventing the implementation of public health and social precautions taken so as to prevent the pandemic. It was concluded that the successful implementation of isolation precautions highly depends on the adequacy of health resources. Similarly, Maity et al. [25] used Stochastic Frontier Production model in order to identify the factors that are responsible for the inequalities among the Indian states and compared them in terms of their combats against COVID-19. They found that elderly population, literacy rate and population density criteria affect the number of recovered people. Sayan et al. [26] employed fuzzy TOPSIS and fuzzy The Preference Ranking Organization Method for Enrichment Evaluation (PROMETHEE) methods to determine the most effective COVID-19 diagnosis methods. Their study concluded that the most effective method was chest computed tomography. Hezam et al. [27] carried out research on vaccination priority and suggested that health staff, high risk groups, elderly people, pregnant and breastfeeding women should be given priority in vaccination practices. Boyaci [28] determined which OECD country was more advantageous in combatting with COVID-19 pandemic by employing SWARA, TOPSIS, Additive Ratio Assessment (ARAS) and Complex Proportional Assessment (COPRAS) methods. According to the findings, Japan was the best country in the efficiency ranking while Chili was the worst one. In addition, Ghorui et al. [29] determined the dominant risk factor for the spread of COVID-19 using Fuzzy Analytical Hierarchical Process (AHP), Fuzzy Hesitant Fuzzy Sets (HFS) and TOPSIS methods. The study concluded that long period of contacts with infected people was the most important risk factor, which was followed by the spreads in hospital and clinic environments. Talip [30] evaluated the course of combat against the pandemic in a group of countries by using Entropy and Weighted Aggregated Sum Product Assessment (WASPAS) methods. According to the results of Entropy method, the most important criterion was "health expenditures". WASPAS method findings showed that Russia, Germany, Canada, the USA, Austria and Swiss were the most successful countries in their combat against the pandemic.
DEA is the most common method employed for efficiency analyses regarding the COVID-19 pandemic in the related literature. Md Hamzah et al. [31] used DEA (Data Envelopment Analysis) method to evaluate the efficiency of the Malaysian health system during the COVID-19 pandemic and found that the majority of states in Malaysia successfully combatted the pandemic. The reason lying behind this achievement was the fair distribution of resources among the states. Ergülen et al. [32] evaluated the efficiency of Turkey during the COVID-19 pandemic using the DEA method. The findings showed that Turkey took the right and effective steps as part of its fight against the pandemic; however, the monthly analyses indicated a fluctuating performance. Shirouyehzad et al. [33] used the DEA method to assess the performances of the countries that have been terribly affected by the pandemic. According to the findings, Singapore, Vietnam, and Belgium were the most efficient countries. Although Singapore is the most densely populated country in Southeast Asia, it was far better than other countries in terms of efficiency. The most efficient country in Europe was found to be Belgium while Italy was the least efficient country. Similarly, Breitenbach et al. [34] evaluated the countries having the highest number of COVID-19 cases in the first 100 days of the pandemic by using DEA method. It was found that Italy, France and Belgium were affected by the pandemic the most. In addition, the efficiency performances of Germany, the USA, Canada and Austria ranged only between 50% and 60% although they are among the prosperous countries. Selamzade and Ozdemir [35] analyzed OECD countries' efficiency in their combat against COVID-19 pandemic using DEA method. The study reported Slovakia and Island as the countries with the highest efficiency score. Italy, Spain, the USA, the United Kingdom and France were the least efficient countries. Mariano et al. [36] used DEA method to evaluate the efficiency of Brazilian states in terms of their fight against COVID-19. The federative unit with the lowest efficiency score was Amazonas, and Manaus had the lowest capital. Bayram and Yurtsever [37] assessed European countries' efficiency during COVID-19 pandemic by employing DEA method. They concluded that Luxembourg was inefficient in terms of virus infection control and Denmark and Island were relatively more advanced than other countries in medical treatment practices. Baş Kaman et al. [38] used DEA method to assess the efficiency of health staff in 9 OECD countries that have been affected by the pandemic the most. According to the results, Czechia, Poland and Portugal were the most effective countries while Italy was at the bottom of the list. Bagrıçak [39] evaluated Turkey's efficiency by comparing the country to OECD and European countries with regards to their fight against the pandemic. The findings showed that Chili had the lowest efficiency score while Ireland was found to be the most efficient country. Turkey was among the countries that failed to be a technically efficient country. Sel [40] employed DEA method to examine how G20 countries' efficiency in health services progressed. According to the results of the analyses, the USA, Argentine, Brazil, China, France, England and Turkey were efficient countries. Finally, Taherinezhad and Alinezhad [41] analyzed the performances of countries during the pandemic by using DEA and Machine Learning methods. DEA by Mohanta et al. [42] measured the performance of 32 states and union territories (UT) of India against COVID-19.
In addition, evaluating the effectiveness of the health system, the effectiveness score obtained with DEA was compared with different models used in many articles. According to the results of the study, 16(50%) of 32 states and UT in India were found to be productive. Among the efficient DMUs, Chandigarh is the most efficient unit while Meghalaya is the most inefficient. Rajasthan was the most applied state for unproductive states.
Sotoudeh-Anvari [43], stated in a literature review that MCDM methods are becoming increasingly popular in modeling COVID-19 problems. It has been determined that 35 countries contribute to multidisciplinary studies, and India is the leading country in this field, followed by Turkey and China. The results show that AHP (including fuzzy AHP) was the most popular MCDM method applied in 37.5% of the articles, followed by TOPSIS and VIšeKriterijumska Optimizacija I Kompromisno Resenje (VIKOR). This study revealed that the use of MCDM methods is one of the most attractive research areas in the field of COVID-19. In addition, 69% of the articles were found to combine various fuzzy sets with MCDM methods to overcome the uncertainty problem when analyzing information.
In recent studies; it is seen that MCDM methods are used in different study subjects related to COVID-19. For example, Pan et al. [44]; A Multi-Criteria Decision Making (MCDM) model was created to evaluate the government's performance against COVID-19. Analytical Hierarchy Process (AHP), Entropy, and TOPSIS method were used to determine the performance of the public health system. In the study, it was stated that both subjective and objective weighting methods should be used for a more accurate assessment. The composite scores of the public health system were determined based on the performance and sustainability evaluation. South Korea, Japan, Germany, Australia, and China received high composite scores; The USA, Indonesia, Egypt, South Africa, and Brazil scored lower. Ahmad et al. [45] analyzed the impact of the COVID-19 outbreak on people's psychological health using Fuzzy Best Worst Method (F-BWM) and Fuzzy TOPSIS methods. Consistency of the results was ensured by comparing the obtained ranks with the ranks found using the Fuzzy WSM and Fuzzy MABAC methods. The results of this study revealed that the five psychological factors caused by the COVID-19 pandemic are anxiety, stress, panic attacks, frustration, and insomnia. Aljaghoub et al. [46] proposed a comparative analysis of various oxygen production techniques using TOPSIS, Multiplicative Exponential Weighting (MEW), Simple Additive Weighting (SAW), Entropy Weight Method (EWM), CRITIC, and the Stochastic Dominance (SD) methods. The results show that oxygen production using membrane technologies is the optimal technique based on all the provided tools and criteria. Ç etinkaya et al. [47], a five-step approach was created using Geographic Information System (GIS) and EWM to determine the mass vaccination site. Cetinkaya et al. [47] proposed a five-step approach using GIS, EWM and the multiple attribute utility theory (MAUT) to determine the mass vaccination site. To assess the state of the Transport infrastructure in terms of COVID-19, Liu et al. [48] created a comprehensive multi-criteria assessment model based on q-rung orthopair, 2-bundle language clusters. Policy implications are presented for transport infrastructure resilience and appropriate model development.

Multi Dimensional Integrated Efficiency Analysis model (MDIEA)
This section introduces the multidimensional integrated productivity analysis model, whose flow diagram is shown in Figure 2. This model is used to determine the performance of countries in the fight against an pandemic and to perform these efficiency analyses.  Table 1.

Stage 1. Determining health indicators as input and output criteria
The first step here is to determine the health indicators that might be taken into consideration while combatting the pandemic. Then, these indicators are divided into two groups as input and output indicators and labeled as input and output criteria.

Stage 2. Calculating integrated weights of input and output criteria
This stage involves calculating the subjective and objective weights of input and output criteria by employing CRITIC and Fuzzy SWARA methods. Later, the weights obtained are combined using SOWIA method.
Step 2.1. Calculating criteria weights using CRITIC method CRITIC is an objective weighting approach developed by Diakoulaki et al. [8] in 1995 that gives an objective result by making use of real values of input and output criteria and their correlation and standard deviation values without taking the opinions of decision makers into consideration. The weights obtained by using this method involved the conflict between criteria as well as contrast intensity of each criterion [49].
Firstly, input and output matrices are created using equation (1) for the input and output criteria. The input matrix is named as and the output matrix as . Each row ( ) in the input matrix ( ) represents the alternatives and each column ( ) represents the input criteria.

Variable Definition
The value of th criterion according to th alternative The value of th criterion according to th alternative max , min The highest and the lowest values of th input criterion The normalized value of The correlation between th and th input criterion pair The total amount of information in th input criterion The total amount of information in th output criterion The standard deviation for th input criterion The standard deviation for th output criterion The weight value of th input criterion according to CRITIC The weight value of th output criterion according to CRITIC The level of importance for th input criterioñ The level of importance for th output criterioñ The coefficient value for th input criterioñ The coefficient value for th output criterioñ The fuzzy weight value for th input criterioñ The fuzzy weight value for th output criterioñ The fuzzy weight value for th input criterion according to Fuzzy SWARÃ The fuzzy weight value for th output criterion according to Fuzzy SWARA The integrated weight value of th input criterion The integrated weight value of th output criterion The th distance value of th input criterion The th distance value of th output criterion The ETWOS method efficiency score of th alternative The unscaled input preference index of th alternativė The scaled input preference index of th alternative The unscaled output preference index of th alternativė The scaled output preference index of th alternative The OCRA method general preference index of th alternative The th index of decision-making units (˜) The alpha cut lower bound value of th input criterion according to th alternative (˜) The alpha cut upper bound value of th input criterion according to th alternative (˜) The alpha cut lower bound value of th input criterion according to th alternative (˜) The alpha cut upper bound value of th input criterion according to th alternative ( ) The upper bound efficiency value of th alternative ( ) The lower bound efficiency value of th alternative The value used while calculating efficiency value ( = 0, . . . , ), ( ) The mid points of decision-making units (1/2 ( + )) ( ) The widths of decision-making units (1/2 ( + )) ( ) The maximum efficiency loss of th alternative The ranking weight of th alternative according to th MCDM approach The normalized value of expresses the weight value of the approach PSI refers to the relative distance of the alternative from the ideal reference point Then, the initial decision matrix [ ] is normalized by using equations (2) and (3) respectively for benefit and cost criteria of input criteria and the normalized input matrix [ ] is obtained as shown in equation (4).
The degrees of relationship between the criteria are determined by calculating the correlation between any two input criteria by utilizing equation (5).
The criteria weights for input criteria are calculated using equations (6)-(8).
The same procedure is followed for the output criteria. The criteria weights for output criteria are calculated using equation (9).
Step 2.2. Calculating criteria weights using fuzzy SWARA method Fuzzy SWARA method, developed by Keršulıene et al. [9] enables the vague and complex evaluation process to be performed more effectively and as realistically as possible.
First of all, the input criteria are ranked by decision makers in a way ranging from the most important to the least important. Table 2 is used to compare th criterion according to − 1th criterion, which has higher level of importance so that the relative importance levels of the criteria could be calculated.
= (˜,˜,˜) represents the level of importance for input criteria and˜= (˜,˜,˜) the level of importance for output criteria. Then, the same procedure is followed for the output criteria.
Equation (10) is utilized to calculate the coefficient value (˜) for input criteria. Next, fuzzy weight values (˜) are calculated using equation (11). Similarly, the coefficient value (˜) and fuzzy weight values (˜) for output criteria are determined.˜= Finally, the relative fuzzy weight value of input criteria is calculated using equation (12) and fuzzy weight value is converted to non-fuzzy deterministic values using equation (13). Similarly,˜and is calculated.
Step 2.3. Calculating integrated criteria weights using SOWIA method SOWIA method, developed by Zaher et al. [10] in 2018, is based on the combination of weights obtained by employing subjective and objective weighting methods. The objective criteria weights ( ) obtained by using CRITIC method and the subjective criteria weights ( ) obtained by using Fuzzy SWARA method are calculated for input criteria. The same procedure is followed to find objective criteria weights ( ) and subjective criteria weights ( ) for output criteria. Finally, the integrated criteria weights ( ) and ( ) are obtained in equation (14) for input criteria and in equation (15) for output criteria by combining objective and subjective criteria weights. = value is known as "factor decision weight" and it is used to indicate the weight of the objective and subjective methods used in the SOWIA method on the result. The decision maker determines the value himself. It can take a value between 0 and 1. Giving a value of 0.5 ensures that the weights of the subjective and objective methods used in the SOWIA method on the result are equal.

Stage 3. Determining efficiency ranking of countries
Efficiency rankings are determined separately for each country by using EATWOS, OCRA and FDEA methods as well as the integrated criteria weights.
Step 3.1. Determining efficiency ranking by using EATWOS method EATWOS method introduced by Peters and Zelewski [11] in 2006, is an efficiency measurement method that allows decision makers to find satisfactory solutions for alternatives in addition to measuring maximum efficiency for input and output variables, which makes it different from the similar efficiency measurement methods [14]. It is also possible to employ this method without taking satisfaction levels into account.
Efficiency scores for countries are calculated by EATWOS method using the integrated criteria weights obtained in Step 2.3 and taking satisfaction levels of input and output criteria into account. The input matrix ( ) shown in equation (1) is normalized using equation (16), and the normalized input matrix shown in equation (4) is created.
Efficiency analysis methods are often based on maximizing output criteria; however, maximizing number of cases and death toll -i.e., output criteria -is not a desirable result. Therefore, shown in equation (1) a new matrix was created by taking the reciprocal of the output criteria (here, number of cases and deaths) in the standard decision matrix (1/ 2 , 1/ 3 ). This matrix was used for EATWOS, OCRA and FDEA in the next steps.
In the normalized input criteria matrix ( ), the minimum normalized input criteria * is determined for each input criterion by utilizing equation (17) and by taking the column vectors ( − → ) into account.
The distance for input criteria is calculated using equation (18) and by taking the amounts of minimum normalized input ( * ) into account.
The distance values obtained for input and output criteria are used for the calculation of efficiency scores. High efficiency score indicates high levels of productivity. The efficiency score results are sorted descending; i.e., from the highest to the lowest. Efficiency score is calculated using equation (19). The alternative with an efficiency score of 1 is labeled as "technically efficient". The farther the efficiency score is from 1, the lower the efficiency level of the alternative is.
Step 3.2. Determining efficiency ranking by using OCRA method OCRA method, introduced by Parkan [12] in 1994, is a nonparametric performance evaluation method successfully employed in many sectors and fields. This user-friendly and practical method can efficiently analyze sensitivities in input and output variables when there are more than one input and output variable.
OCRA method is employed to calculate efficiency scores of countries by making use of the integrated criteria weights obtained in Step 2.3. The unscaled input indices are calculated in equation (20) The scaled input preference indices are calculated by utilizing equation (21).
Finally, the scaled general preference index is calculated utilizing equation (22) and sorted descending. The decision unit in the first rank is labeled as the best one.
Step 3.3. Determining efficiency ranking by using FDEA method Since it is a efficiency measurement method using the FDEA database, it is highly recommended that the input-output data be reliable and carefully selected. In addition, it is more appropriate to use FDEA models developed to measure efficiency, since it is difficult to reach full and precise input-output data in real-life problems when the data related to real-life problems are not fully known [13]. Kao and Liu [51] model, which is an FDEA method used for the limited data and the data with known precise values, was employed and the models created were analyzed using LINGO 17 software. The Kao-Liu model was developed by converting the fuzzy data envelopment model into the traditional data envelopment model by using Zadeh's extension principle and cut method.
The normalized input matrix shown in equation (4) is weighted by utilizing equation (23), and the weighted normalized input matrix shown in equation (24) is created accordingly.
Similary, the normalized output matrix is weighted by utilizing equation (25) and the weighted normalized input matrix shown in equation (26) (27).
The lower efficiency bounds of˜input and˜output criteria data are calculated as in equations (28)- (30).

Minimax regret approach
Efficiency values of alternatives in FDEA models are defined for certain intervals. These values might not be compared since their centers are the same although the widths of value ranges vary. Such a situation requires a technique to be used while ranking and comparing efficiency values [52]. To achieve this purpose, Wang et al. [52] developed minimax regret ranking approach. This approach can be employed to rank and compare efficiency ranges of DMUs even if they have different widths but equal centers [53].
According to minimax regret approach; assume that efficiency range of "n" number of alternatives are defined as in equation (39) Assume that = [ , ] has been selected as the optimum efficiency range and = maks ̸ = { }. maks, represents the highest value. maks ( ), means maximum efficiency loss.
If < , decision maker faces efficiency loss and feels regretful. In this case, maximum efficiency loss is represented as in equation (40).
If ≥ , no efficiency loss occurs for decision maker and the regret is defined as "0". Equation (41) is formed when these two situations are taken into account. If ≥ Maximum efficiency loss for each efficiency range is represented in equation (42) Here, the efficiency range with the lowest maximum efficiency loss is the most convenient efficiency range.

Stage 4. Determining integrated efficiency ranking by using TPOP method
At this stage, TPOP method is used in order to obtain one single ranking by combining efficiency rankings determined by employing EATWOS, OCRA and FDEA methods.
TPOP is an advanced version of ENTROPY and exponential weighting approaches which give acceptable relative weights [54]. It was developed to form a common ranking out of different rankings obtained by utilizing different MCDM approaches. This method is based on the reference point theory. The alternative with a minimum distance from the reference point ranks the first and is selected as the best alternative. On the other hand, the alternative with a maximum distance from the reference point ranks the last and is selected as the worst alternative [15].
In the first step, [ ] matrix is created as in equation (43), and this matrix includes rankings obtained using different MCDM methods.   . . .
Entropy ( ) in TPOP method is utilized in order to remove the uncertainties existing in decision making problems regarding the data and to obtain more precise results. It is calculated using equation (45). , which is calculated using equation (46), allows comparisons among the methods regardless of selection procedures depending on rankings within themselves.
′ parameter is calculated using equation (47). 1 − can be considered a complement to the entropy of final decision values determined through th approach.
′ parameter value is calculated using equation (48). ′ value in the equation should be within ≤ ′ ≤ 2 range and the number of the method to be applied should be ≥ 2 Precise weight values of MCDM methods are calculated using equations (49)- (51). The sum of these weights should be equal to 1 ∑︀ The sum of all values is equal to maximum 1. Accordingly, max ∑︀  (50) and (51). The precise weight values calculated after the equations are defined become 1 + √ ≤ ≤ 2 +1 . The weights are normalized using equations (52) and (53). While , refers to th MCDM method in which the best ranking is ascending; , is the alternative assessment method in which the best ranking for th MCDM method is descending.
Equation (54) is used to obtain exponential weighted normalized ranking value for (ℎ ) each alternative.
Final decision index for alternatives is calculated using equation (55).
The calculated PSI values are taken into consideration in the ranking phase. The alternatives are ranked descending; maximum PSI being at the top.

Application
This study analyzed the efficiency of 34 OECD countries displayed in Table 5 during COVID-19 pandemic. Colombia, Costa Rica, Hungary and Israel were excluded from the study since the researchers could not access the data regarding some of the input and output criteria.

Stage 1. Determining health indicators as input and output criteria
First, the related studies in the literature were examined in order to create an input and output criteria list. Next, five decision makers, who are doctors, nurses, academics, examine this list, and the input and output criteria were finalized as shown in Table 3. The data for the input criteria were obtained from COVID-19 statistics in OECD [9], Turkey Health Statistics Report [55], World Bank [56] and Our World in Data [57]. The data regarding output criteria were obtained from Worldometers COVID-19 statistics [58].
The data of the input variables cover the data between 2016 and 2020. It is not possible to obtain data for all countries as of the same date. For this reason, the dates of data collection may vary from country to country. Externally, all of the data for the rural population ratio variable belongs to 2020, and all of the data for the diabetes rate variable belongs to 2019. The data on the number of vaccines and the number of tests were obtained on October 26, 2021. All of the data for the output variables were obtained on October 26, 2021.
Urban population ratio, the prevalence rate of diabetes, the population of people over 65 years old, the number of beds in intensive care units, and the number of vaccination doses are the study-specific criteria that are different from other studies.
Rural population ratio, diabetes prevalence rate, the proportion of the population over 65 years old, number of intensive care beds and number of vaccine doses are study-specific criteria, unlike other studies.
Among the criteria given in Table 3, 10 , 11 , 1 , 2 , 3 indicators directly affect the COVID-19, while 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 indicators indirectly affect the COVID-19. According to the results displayed in Table 3, the average number of infected cases per one million people in 34 OECD countries is 84.467. New Zealand has a quite lower number of infected cases than the average and the lowest death cases and the lowest number of recovered cases. Czechia has the highest number of Infected COVID-19 cases. Turkey has the lowest ratio of doctors and nurses per 1000 people as well as the lowest percentage of health expenditures in the Gross National Product. However, Turkey has the highest number of intensive care unit beds, which is 48 beds per 100.000 people. Only 9 countries (Turkey, the USA, Germany, Austria, Belgium, France, Canada, South Korea, Luxemburg) along with Turkey have intensive care bed capacity that is higher than the average.

Stage 2. Calculating integrated weights of input and output criteria
In CRITIC method, values are calculated by utilizing equations (1)-(9) while equations (10)-(13) are used to calculate values in Fuzzy SWARA method. The researchers consulted 5 experts while determining the weights using Fuzzy SWARA method. The weights determined using CRITIC method are completely based on objective values. Then, equations (14) and (15) are used to obtain values from the SOWIA method. The weight values obtained from all three methods are displayed in Table 4.
According to the results of CRITIC method displayed in Table 4, 9 (population density) is the input criterion that has the most important weight while 1 (number of recovered people) is the most important output criterion. As for the Fuzzy SWARA method, 4 (intensive care bed capacity) is the most important input criterion and 3 (number of COVID-19 cases) is the output criterion with the highest level of importance.  Table 4. Weight values of criteria. According to the combined results, the most important output criterion is 3 (number of COVID-19 cases) and the most important input criterion is 4 (number of intensive care beds).

Stage 3. Determining efficiency ranking of countries
In this stage, efficiency analysis for 34 OECD countries was performed by using EATWOS, OCRA, and FDEA methods and following Steps 3.1-3.3. The results are presented in Table 5. The analysis was done according to three different models: The model without satisfaction level (EATWOS-NS); the model with satisfaction level for the output criterion "number of recovered people" (EATWOS ( 1 ); the model with satisfaction level for the output criteria "number of cases and people who recovered and died" (EATWOS ( 1 -2 -3 )). The mean Table 5. Recommendations for improvement for inactive countries.      The results are displayed in Table A.3. In Figure 6, the results of the TPOP method obtained from scenarios are compared. As seen in Figure 6, New Zealand is the only technically efficient country according to three different scenarios. The reason lying behind this result might be that New Zealand had the lowest number of people infected with COVID-19 -thus the lowest death toll -when the data were obtained. Chili and Finland rank at the bottom of the list. According to scenario 1 -i.e., the results for which satisfaction levels were not determined -, the average efficiency value of OECD countries is 52.7%. Czechia, Estonia, Latvia, Poland, Slovakia, and Slovenia are the countries that have scores above the average value. Nevertheless, this situation was not good enough for their fight against the pandemic. Australia has the lowest population density; however, its efficiency score is lower than the average. Chili, which is the last country in the efficiency ranking, has the highest number of vaccination doses per million per capita. In addition, the number of health staff and the number of beds are lower than average while the number of COVID-19 cases and the death toll is above average. It might be suggested that the country should increase the number of health staff/necessary equipment and make some changes in precautions taken or take stricter precautions. According to Scenario 2 and Scenario 3, the efficiency score averages of the countries were 55% and 57%, respectively (Tab. A.3).

Results and discussion
In this section, improvement suggestions are presented for OECD countries that are not effective in the fight against the COVID-19. Table 5 presents the approximate values of output criteria that the countries which failed to obtain the necessary efficiency scores with their available health workers should achieve in order to be an efficient countries. For the EATWOS method, the EATWOS(NS) model, that is, the model in which satisfaction levels are ignored, was taken into account.
According to the results displayed in Table 5, Chili, which is the worst country in terms of efficiency with its rate of 54%, should decrease the number of COVID-19 infected people from nearly 87.000 to nearly 39.000. Also, Turkey, a country falling behind the bound efficiency score, can be an efficient country when the number of people who are infected with and die of COVID-19 decreases by 51% and the number of people who recover increases by 105%. The USA should lower the number of COVID-19 infected people per million from 139.000 to 72.000 so that it can be an efficient country. Czechia, which is second only to New Zealand, needs to increase the percentage of patients who recovered from COVID-19 by 83% to become an efficient country. When the percentages of recovered patients are examined, it is seen that all the countries except Mexico and Slovakia are quite far from being technically efficient countries.
Countries can take a number of measures in the long term to achieve the target values given in Table 5  The date of data collection and the methods and criteria used in this study differ from other studies in the literature. For this reason, it is natural that the results obtained as a result of the study differ from the results obtained from other studies. However, other studies have offered similar recommendations to the abovementioned recommendations to prevent the spread of the pandemic. According to Shah et al. [59]; Reducing the rate of transmission and enforcing rules and regulations for prevention will have the most beneficial effect in controlling or slowing the spread of COVID-19. stated that minimizing contact with other people was a good measure to overcome the dreaded infection and had good results. Razzaq et al. [60] stated that awareness of the use of medical masks, social distancing, frequent use of disinfectants or hand cleaning, and supportive care during treatment are the strategies followed worldwide in this struggle. Ultimately, social distancing and supportive care of the infected are found to be significant in decreasing the basic reproduction number more rapidly. In the pandemic situation, predicting and analyzing whether a patient is suffering from the virus that causes COVID-19 disease will facilitate the testing process. Therefore, predicting these viruses can be beneficial for researchers by obtaining full identification and important symptoms, and can ensure that patients are vaccinated as soon as possible (Jain et al. [61] and Vedika et al. [62] stated assessing the impact of the COVID-19 emergency on social orders, economies, and vulnerable groups is crucial for making and adapting recommendations for governments and their accomplices to get out of the emergency and ensure no one is abandoned in this effort.

Conclusion
In the current study, a new multi-dimensional integrated efficiency analysis model was developed to determine the efficiency of OECD countries in their fight against COVID-19. The proposed model can be employed to evaluate the efficiency of countries during any pandemic. The model was specifically applied to the COVID-19 pandemic in this study. The literature review revealed some studies using different analyses regarding COVID-19. The current study is expected to contribute to the literature due to its criteria variety and the methods employed together.
When all the results are evaluated, it is difficult to argue that a single criterion is effective in the fight against the pandemic. For an effective fight against the pandemic, it is of great importance to keep the rate of transmission, and therefore the number of infected people, under control. The results might vary from those of other studies due to the use of different methods and the time of practice. It is not possible to obtain data regarding all OECD countries and all the criteria for the same date and to access up-to-date data, which might be considered the limitation of the current study.
Future studies might use different input and output criteria. In addition, multiple analyses might be performed by making some changes between input and output criteria. For instance, several tests and several vaccination doses might be taken as output criteria instead of input criteria. Different MCDM methods can be used and the course of the pandemic might be compared by performing efficiency analyses for different periods.
Appendix A.