Good Governance Measurement using Rotberg's Output-Based Approach for Kyrgyzstan

This study evaluates the concept of good governance by comparing traditional input-based measures with a novel output-based methodology. Indices like the Worldwide Governance Indicators (WGI) are widely used for cross-country comparisons but face criticisms for their reliance on input and capacity-based approaches, which may not accurately reflect governance outcomes experienced by citizens. Rotberg (2016) proposed an output-centred method that assesses governance quality through tangible political goods provisions, utilizing a bottom-up approach. This research applies Rotberg’s methodology to measure good governance in Kyrgyzstan using data on 36 indexes over a period from 1991 to 2019. Missing values were imputed using Principal Component Analysis. Comparative analysis of Rotberg’s index with the WGI reveals divergent trends, explained by differences in methodology and underlying data. The research findings highlight the need for the design of a rigorous methodology for the computation of the output-based good governance index for developing countries. Addressing the identified issues will facilitate the widespread adoption and refinement of this alternative measure, fostering better governance practices worldwide.


Introduction
The concept of good governance is multifaceted, encompassing diverse interpretations and measurement frameworks.While various indices attempt to assess governance quality, such as the Worldwide Governance Indicators (WGI) (Kaufmann & Kraay, 2023), criticisms have been raised regarding their input and capacity-based approaches and comprehensiveness.Despite these limitations, the WGI is widely used in meaningful cross-country and over-time comparisons, being the only available open-source governance quality data today.
It measures the quality of national governance by aggregating various indices of governmental effectiveness, regulatory quality, stability, and control of corruptionattributes that can be estimated through expert surveys but are challenging to quantify using nationally generated statistics.These indicators are predominantly normative, reflecting policy preferences rather than measuring the fulfilment of citizen-requested priorities (Rotberg, 2014;2018).Rothstein and Teorell (2013) argue that the World Bank's focus on the input side of the governance equation "makes it impossible" to accurately assess governmental performance.This critique highlights that while input-based measures are valuable for evaluating institutional readiness and policy frameworks, they fail to adequately capture the actual outcomes experienced by citizens, which are essential for a thorough assessment of governance effectiveness.
An alternative methodology was proposed by Rotberg (2016), advocating for the assessment of governance quality through the lens of 'outcomes' or 'outputs' utilizing a quality-centred approach employing a bottom-up methodology.The author posits that this approach is better suited for gauging approximate levels of tangible service provisions (governance) rather than subjectively rating nations based on perceived operational quality, impartiality, bureaucratic autonomy, or ability to influence citizens through persuasion or coercion.
The principal constraint of employing this alternative approach to assess good governance lies in the lack of readily available data for download, coupled with the non-calculation of the index for various countries.Consequently, this study endeavours to compute the measure of good governance for a specific developing country, Kyrgyzstan, utilizing an output quality-based methodology to bridge this gap.
Given that the output-based good governance index can serve as a robust diagnostic tool for civil society, donors, and government due to lower reliance on subjective perception-based estimations compared to input-based indexes, the discussion of prospects of the novel measure of the governance quality and practical aspects of its application for cross-country analysis is relevant and timely.
Rotberg's approach was applied to compute the 2009 Index of African Governance for 53 African countries to measure their ability to provide political goods to inhabitants (Rotberg & Gisselquist, 2009).This paper is the first attempt to measure good governance for a country beyond the African continent to estimate its potential for global application.The choice of Kyrgyzstan for the good governance measurement in accordance with Rotberg's approach was pre-determined by the availability of accurate and valid statistical data used for the estimation of proxies for social and economic development compared to other central Asian states such as Uzbekistan and Turkmenistan.Moreover, experts in Kyrgyzstan, a country with relatively more democratic political regimes (Freedom House, 2016) compared to other countries in the region, could more openly express their views and opinions, which underlie the indices used for computation of the output-based index of good governance.Therefore, the index computed for the Kyrgyz Republic is less likely to suffer from biases and measurement errors of original data.

Literature Review
Good Governance is a multifaceted concept subject to diverse interpretations and definitions.Its comprehension spans a multitude of proprietors and encompasses various explanatory and measurement frameworks.
A producer of Worldwide Governance Indicators, Kaufmann et al. (2011), define governance as the traditions and institutions by which authority in a country is exercised.This includes the process by which governments are selected, monitored, and replaced; the capacity of government to effectively formulate and implement sound policies; and the respect of citizens and the state for the institutions that govern economic and social interactions among them.All three areas are assessed on the basis of input and perceptions-based governance data.
Kauffmann's definition of governance is criticized as excessively broad and disconnected from normative political theories about justice, arguing that it overlooks the differentiation between issues of power access and its exercise, while also aligning good governance at the national level with impartiality, highlighting the imperative for governing institutions and officials to refrain from exhibiting favouritism in the implementation of public authority (Rothstein 2011, Rothstein & Teorell, 2013).Fukuyama (2013) posits that expert surveys used in WGI possess inherent weaknesses.The absence of a shared understanding among experts regarding terms such as "governance" or "regulatory effectiveness" leads to responses being provided honestly but from divergent viewpoints.Hence, what one individual perceives as corruption may be interpreted by another as reciprocal gift-giving or the rule of law can be interpreted or used synonymously with property rights.Systematic biases are considered to be the main limitation of the perception-based data, because respondents differ systematically in their perceptions of the same underlying reality, and ideological orientation biases in the organization may provide a subjective assessment of governance (Kurtz & Shrank 2007).In addition, the phenomenon of "groupthink" emerges, wherein experts tend to be influenced by one another and are generally hesitant to provide assessments diverging from those of their peers, often due to apprehensions of potential criticism (Charron, 2021).
Despite the presence of subjective margins of error, the WGI is widely used in meaningful cross-country and over-time comparisons being the only available opensource of governance quality data today.Fukuyama (2013) evaluates the existing measures of governance as highly inadequate stating the fact that if the quality of government is a mixture of capacity and autonomy, governments are complex collections of organizations why single sovereign nations are treated as the unit of analysis as governance varies enormously within countries, both by specific function and by region and governance problems cannot be assessed in one level many issues may occur in the interactions between levels of governments.He excludes democratic accountability stating that democracy and good governance are empirically supportive but there is more of a theory and the connection is not demonstrable empirically if it is defined as one to the other.He proposes four broad approaches to evaluate the quality of governance: procedural measures, capacity measures, output measures, and measures of bureaucratic autonomy.To define procedural measures Max Weber's "Weberian bureaucracy" is used as an ideal type with 10 conditions and considered as a core of any measure of quality governance.As the measurements of state capacity the tax extraction, level of education, and professionalization of government officials are suggested.
Lack of data is considered as the limitation of this approach as the state capacity varies substantially across functions, levels of government, and regions.Fukuyama (2013) states that output measures should not be used as a state quality measure as they cannot be divorced from procedural and normative measures, and it might be better to leave the output as a dependent variable to be explained by state quality rather than being a measure of capacity in itself.Expert surveys are suggested to measure the last measure of bureaucratic autonomy in which experts are asked to measure and evaluate the autonomy of a given bureaucracy, however, it is problematic due to the poorly specified concept of autonomy.
Rotberg (2016) challenges the use of input and capacity-based approaches to measure governance quality which is widely employed by the makers of many indexes, including the WGI.He argues that the indicators used are largely normative, policy preferences oriented, and do not measure the citizen request priorities by defining 'governance as the performance of governments the delivery of the essential political goods to constituents'.He advocates measuring the quality of governance by 'outcomes' or 'outputs' and a quality-based approach with a bottomup method of defining governance that emphasizes results using a five-category matrix that draws on an assessment of 57 variables based on largely objective, publicly available, internationally collected, and sorted datasets.Rotberg advocates a shift from the employment of easier-to-assemble expert opinion surveys to the much harder development of measurement instruments that rely on national statistics.Moreover, he considers that governance is tangible, actionable and exemplifies that if we agree that taxpayers expect their governments to perform in such a manner that citizens will be secure and safe; require a robust rule of law that delivers fair adjudication of disputes between persons; prefer not to be cheated and participate in rule-setting and thus governing themselves, or at least prefer to have a voice in agenda-setting; individuals prefer to prosper, eat better food, to be housed adequately, to be paid fairly for their labour and to believe that they are free to use their skills to better themselves; and citizen's generally expected states to provide educational opportunities, better health care, clean water, a minimally polluted environment then and so on, then it makes sense to measure governance based on the realization of the need.
Based on these examples, he divides fundamental responsibilities and delivery expectations of governments into five categories of analysis: • safety and security, • rule of law and transparency, • participation and respect for human rights, • sustainable economic opportunity • human development which can measure the performance of any government, at any level.
The main strength of outcome-based conceptualization of governance and bottomup method is the employment of objective rather than subjective data where good ministerial attitudes and fine words matter far less than results, and a possibility to apply them for diagnostic purposes to find out achievements and gaps within the national governmental apparatus to prioritize focus and funding allocation.According to the author, the other strength is the concept when performance is measured, performance usually improves, especially when it is measured, and the results are reported.
The weakness of this conceptualization is the reliance on the data of national statistics offices where a lack of appropriate funding and staff may influence accurate data collection.Comparatively, to input-based subjective conceptualization with selection bias and subjectivity, the limitation is poor.
In addition, Rotberg states that any quantitative or qualitative measurement tool that expects to be policy-relevant must do three things: specify the most effective elements to assist policymakers in designing more effective policies, aid policy by helping policymakers think through or analyse problems, identify systematic deviations from optimal decision making and the identification of certain correcting principles.
Literature review shows that in developing countries, input-based measures of good governance can generate biased perceptions and beliefs due to biases and the impartiality of expert opinions.Moreover, the quality of governance in developing countries does not necessarily improve the citizens' quality of life since extractive institutions and corruption may impede the successful implementation of the reforms.Therefore, such input-based indexes as WGI may not reflect the quality of governance in developing countries, welcoming alternative approaches, including Rotberg's approach, which provides the opportunity for a holistic evaluation of good governance.Despite the relevance of this novel approach, to the best of our knowledge, this paper offers the first attempt to measure good governance using Rotberg's methodology outside the African continent.We estimated the good governance index for Kyrgyzstan, one of the developing transitional countries in Central Asia.

Research Methodology
The Governance Indexes in Comparative Perspective Database (Canadian International Development Platform, 2021)  Given the disparate ranges of the selected indices, we normalized them using the following formula: This procedure enabled us to standardize indices within a range from 0 to 1.In cases where an indicator had a reverse impact on good governance, such as the Social Institutions and Gender Index (OECD, 2024), which ranges from 0 (indicating no discrimination) to 100 (indicating absolute inequality), the final score was calculated by subtracting the normalized value from 1.
A significant portion of the indicators contained missing values, posing a challenge to computing the good governance index.To overcome this limitation, we employed principal component analysis and imputed the missing values.Instead of assigning weights to indicators, we utilized the imputations with five components.
The resulting dataset comprised normalized data, excluding several outliers.Indicators with exceptionally low values were considered outliers and thus excluded from the dataset.For instance, indices with low variation over the given period and values close to 0 after normalization were deemed uninformative but could adversely affect the composite index of good governance.For example, the Composite Index of National Capability (Correlates of War, 2024), which ranged between 0.000343 and 0.000394, was omitted, as its inclusion notably lowered the computed Security and Safety index from 0.51 to 0.12.Negative or strictly equal to 0 values were replaced by 0.1, while values exceeding the maximum threshold were assigned 1.This approach simplified the computation of the good governance index.
Finally, we divided the final dataset into five categories corresponding to the five dimensions of good governance outlined in Rotberg's approach.For each category, the composite index was constructed using the widely recognized formula employed in the calculation of the Human Development Index:

Interpretation of the output-based Good Governance indices
The acquired good governance index provides insight into the extent to which the government of Kyrgyzstan has addressed the needs and expectations of its citizens across these five dimensions since 1992.
Based on the calculated indicators (refer to Figure 1), the level of good governance has exhibited gradual improvement since 2011.Exceptions include the Human Development Index (UNDP, 2024), which has steadily declined since 2010, and political participation, which experienced growth followed by a notable decline in 2017-2018.However, a significant improvement in the human development index was observed between 2007 and 2010.Over the past 29 years, the human development index has not undergone considerable changes.
The level of sustainable economic development fluctuated over the period, experiencing a moderate decline from 1991 to 2001, primarily attributed to the transition crisis in Kyrgyzstan, compounded by external shocks such as the default in Russia in 1998, a key economic partner.A slight improvement was observed between 2002 and 2004, which was halted in 2005 due to political instability in the country.The Global Financial Crisis had a significant impact on the sustainable economic development index, plummeting by 27% in 2009, followed by a recovery in the succeeding year due to changes in power dynamics.In summary, Rotberg's good governance index (GGI) reflects major shocks and trends in key development domains: economic, social, and political.The Human Development Index stands out as the highest among the five components of the good governance index, reflecting Kyrgyzstan's relatively high levels of education, healthcare, and social security, while the development of other domains was subject to significant internal and external shocks.

Comparison of WGI and Rotberg's Good Governance indices
Next, we compare the World Governance Indicators with the governance index computed according to Rotberg's output-based approach.However, WGI does not offer a composite governance index.Therefore, for comparative analysis, we synthesized six governance indices into one using Formula 2. However, Rotberg's Index falls short of capturing the impact of shocks that adversely affect governance quality, such as the 2005 revolution.The overall disparity between them can be attributed to differences in the essence of the two approaches: WGI follows an input-based methodology, while Rotberg's approach is output-based.WGI primarily assesses the quality of public administration, policy-making, and democratic institutions without measuring such outcomes as economic and social Even when considering domains common to both WGI and Rotberg's approach, differences persist.For example, the Rule of Law indexes computed in compliance with input-based (WGI) and output-based (Rotberg) approaches (Figure 4) do not entirely overlap but exhibit a shared trend, particularly noticeable post-2010.

Figure 4. Rule of Law: WGI and Rotberg's Good Governance Index
Source: Author's calculation of Rotberg's GGI;World Bank (2024)

Several methodological aspects could potentially explain these differences
Firstly, the methodology of index generation differs.WGI is estimated using the Unobserved Components Model (UCM), while we employed Principal Component Analysis to impute missing values.Consequently, numerically, the indices are expected to generate different results.
However, there are no discrepancies in aggregation, as we followed the same aggregation procedure.The only variation was that WGI indicators were negative for all periods, whereas our formula generates positive values (the product of an even number of negative values).To preserve the accuracy of interpretation, we assigned a minus sign to the obtained composite index.
Another methodological aspect is the difference in the set of indicators used for calculation.We adhered to the methodology proposed by Rotberg.However, initially, this index was proposed for African countries, and therefore, some of the

Conclusion and recommendations for future research
This paper delves into the discourse surrounding good governance measurement, particularly highlighting Rotberg's approach, and presents an empirical examination of its application in Kyrgyzstan.Collecting data on 36 indexes from 1991 to 2019, we calculated the good governance index in compliance with Rotberg's approach for the Kyrgyz Republic.As missing observations were the major computational challenge, we used the Principal Component Analysis to impute missing values.The comparative analysis of the obtained index and WGI shows that they do not demonstrate parallel trends, which can be explained by the composition of the indexes, methodological differences, and data sources.
The output-based approach is deemed promising for shifting governance assessment from a narrow input-based approach to a holistic output-based one.Therefore, for the wide use of this alternative measure, the following issues should be addressed: • The selected indexes should cover all the countries across the world to leave room for cross-country comparisons of governance quality.The set of indexes proposed by Rotberg does not apply to other non-African countries and resulted in a reduction of the sample size from an initial 94 to 36 indexes.
• The methodology of index computation should be meticulously developed.The method used in this paper is not flawless and should be improved significantly.
• The combination of perception-based and objective statistical data will add value to this index.
This research contributes to the ongoing discourse on effective governance measurement and informs policymakers and practitioners striving to enhance governance outcomes in Kyrgyzstan and beyond.
suggests employing 94 indicators to construct the composite output-based index, which encompasses five dimensions of good governance: Human Development, Sustainable Economic Development, Rule of Law and Transparency, Political Participation and Respect for Human Rights, and Security and Safety .However, we opted to use only data on 36 indicators collected from 1991 to 2019 to compute the good governance index for the Kyrgyz Republic.The data sources used for data collection are listed in Appendix 1.The exclusion of the remaining two-thirds of indices stemmed from various factors, including Kyrgyzstan's exclusion from the sample and limitations of the datasets, such as data being available only for clustered years, among others.
The peak of the political participation index improvement was notable in 2001 when Kyrgyzstan re-elected Askar Akaev for a third term.Although the dynamics levelled off in the subsequent four years, the highest decline occurred in 2009, showing a 31.1% decrease, which recovered in 2011 following the second revolution.Since 2012, the overall trend of political participation has remained comparatively stable.The rule of law index displayed significant improvement from 2001 to 2005, leading to advancements in the indicator in 2006.However, in 2007, the index notably decreased due to violations of press freedom.Recovery continued until 2015, with gradual improvement afterward.

Figure 1 .
Figure 1.The dynamics of good governance indices in the Kyrgyz Republic in 1991-2019Source: Author's calculation.

Figure 2 Figure 2 .Figure 3 .
Figure2illustrates a strong correlation between WGI and Rotberg's index, which is particularly evident post-2010.This parallel trend in the two indicators confirms the accuracy of Rotberg's approach in reflecting governance quality in the Kyrgyz Republic.Figure3further highlights a robust positive correlation between the two indices, with a Pearson correlation coefficient of 0.3.
Consequently, while political shocks may depress the WGI index, Rotberg's Index could remain stable as the shock can be moderated by relatively steady indexes, e.g.human development.
were not available for Kyrgyzstan, such as the ACBF-African Capacity Indicators(African Capacity Building Foundation, 2024).Consequently, we could not replace or incorporate additional indicators not available in Rotberg's list of indices.We argue for the necessity of developing a methodology that allows for the estimation of a universal good governance index, facilitating cross-country comparisons.