Augment the SDG indicator framework

The 231 indicators of the Sustainable Development Goals (SDGs) are crucial for measuring progress on 169 targets. But the seemingly large number of indicators and the associated cost of monitoring have led to calls for streamlining. Here I present an alternative perspective on the SDG indicators by paying due regard to their performativity, or ability to shape outcomes. By drawing on the literature on conditions under which indicators have unintended consequences, I argue that there are rather limited, chiefly quantitative indicators to adequately address the multifaceted aspects of diverse targets they serve. The SDG indicators in their current form run the risk of ‘the tail wagging the dog ’ by powering over their targets. Further streamlining the indicator framework would likely exacerbate the risk, and undermine the ambition, integrity, and legitimacy of the SDGs. The analysis recommends augmenting the indicator framework and integrate indicator development into future negotiations of global goals and targets.


Introduction
On 1 January 2016, the world embarked on a 15-year mission to achieve the United Nations' 2030 Agenda for Sustainable Development.The 17 Sustainable Development Goals (SDGs) and their 169 targets have garnered significant public and scholarly attention (Biermann et al., 2022).Yet, a crucial aspect that warrants equal scrutiny is the selection and use of indicators.Indicators serve as the key tools for decision-makers to track progress towards the SDG targets.Therefore, they have a decisive impact on SDG implementation, as well as the ultimate determination of whether the world is closer to realizing the SDGs by 2030.
As of 2023, there are 231 official indicators in use, including some that are controversial such as those based on gross domestic product (GDP) (United Nations, 2017, 2022).Developed and annually reviewed by the Inter-agency and Expert Group on SDG Indicators (IAEG-SDGs), these indicators are now widely deployed at all levels of sustainability governance.National and local governments use the indicators to measure own progress towards sustainable development, which they report in their voluntary national and local reviews.The High-level Political Forum on Sustainable Development in turn surveys these reviews for its annual report on global progress.Some organizations use them to rank the progress countries have made on the SDGs (Shore and Wright, 2015;Schmidt-Traub et al., 2017;Diaz-Sarachaga et al., 2018;Sachs et al., 2021).
As a key means to claim success, the SDG indicators effectively encourage strategic behavior of governments and other target actors.
There is a general 'risk that only what gets measured gets done' (Mac-Feely, 2020: 369), or actors behaving in such a way that the world is improving on paper, but not in reality.The indicators could also discourage the type of transformative change that the 2030 Agenda calls for.For instance, they prioritize existing data that supports the status quo, and could divert resources away from collecting new data needed for change.Such negative consequences of indicators, while not entirely avoidable, should be taken into account when developing and using them.
Yet the scientific discourse on the SDG indicators has remained largely centered around questions about practicality (Mair et al., 2018).A particular concern has been raised over the large number of indicators and the associated cost of SDG monitoring, which is estimated to be in the billions of dollars (Jerven, 2017).Compared to the Millennium Development Goals (MDGs) with 60 indicators, the number of SDG indicators is indeed significant.The increased burden, coupled with the fact that many of them have been found to be inadequate measures of progress, has sparked debate among scholars.For example, van Vuuren et al. (2022: 142) argue that the indicators are 'too many, too broad, unstructured, and sometimes not formulated quantitatively', and call for a streamlining of the indicators with fewer, more relevant and specific indicators.This view, also shared by others in the field (e.g., Lyytimäki, 2019;Kubiszewski et al., 2021), is what this Perspective aims to critically engage with.
Here I present an alternative perspective by paying due regard to the E-mail address: r.kim@uu.nl.As I will show, the analysis suggests that the SDG indicator framework in its current form runs the risk of 'the tail wagging the dog', or prioritizing means over ends.Streamlining the indicator framework without proper considerations could exacerbate this risk, and result in a loss of ambition, integrity, and legitimacy of the 2030 Agenda and the SDGs.One way to mitigate the risk would be to streamline the entire SDG framework by reducing the number of targets.If the current number of targets were to be justified, however, the indicator framework may even need to be expanded and diversified.
The discussion is organized as follows.I begin by reviewing the literature on sustainability indicators at the science-policy interface, with a focus on the concept of performativity.Then I draw on relevant literature, specifically Goodhart's law and Campbell's law, to provide a critical appraisal of the current SDG indicator framework.Finally, I make a case for augmenting the indicator framework, and discuss implications for the future of governance through global goals.

Indicators are performative
A key concern when selecting and using indicators is their relevance, or how adequately indicators measure the monitored phenomena of interest (Hák et al., 2016).Many scholars have critiqued the SDG indicators from this perspective, often concluding that the indicator framework should be streamlined with fewer but more relevant indicators.For example, Kubiszewski et al. (2021: 146) argue that 'most of the current indicators are not necessary' because they are 'unable to measure sustainable development holistically'.The authors demonstrate that a small subset of the indicators, specifically eight out of 231, are sufficient to measure progress on improving life satisfaction.
Pursuing a limited number of relevant indicators may seem costeffective, but it is important to exercise caution as it presumes a linear relationship between science and policy.Under this assumption, indicators are often viewed as apolitical and impartial measurement tools with little impact on how goals and targets are interpreted or implemented (Hardi et al., 1997).The process of indicator development is often divorced from policy considerations, with decision-makers first agreeing on goals and targets to be measured, and experts subsequently developing indicators to track progress (Rametsteiner et al., 2011).In the case of the SDGs, the choice of indicators was delegated to statisticians who met behind closed doors after the goals and targets were established (Kamau et al., 2018;Kapto, 2019;MacFeely, 2020).
According to science and technology studies, however, the clear division between science and policy does not exist in practice (Watson, 2005).Instead, there is a blurred boundary area between the production and use of scientific knowledge.Indicators in particular play a critical role at this science-policy interface, serving as boundary objects that travel back and forth between the two domains (McCool and Stankey, 2004;Turnhout et al., 2007;Turnhout, 2009;Star, 2010).This has been observed in relation to the SDGs where, contrary to the assumption, the selection of indicators was never free from politics (Fukuda-Parr and McNeill, 2019).Statisticians received instructions from their governments, and the interests of powerful governments had a significant influence over the indicator selection process (Kapto, 2019).
What this means is that indicators are not simply passive tools that describe social realities; they also function as a form of social action (Espeland and Sauder, 2007).Indicators are performative in nature.They shape and direct the actions of those who use them, as well as the phenomena they measure.For instance, development indicators redefine development as specific measurable accomplishments, thereby influencing the type of development that would ensue (Merry, 2019).This can lead to more funding for projects that yield measurable changes, while unmeasurable development needs are overlooked.Such consequences of indicators are not always unintended and may sometimes be deliberately crafted by certain political interests during indicator selection.
The performativity of the SDG indicators has been examined by social scientists (e.g., Adams, 2019;Fukuda-Parr, 2019;Fukuda-Parr and McNeill, 2019;Kapto, 2019;Merry, 2019;Razavi, 2019;Unterhalter, 2019;Yamin, 2019).These studies have found that the indicators have a distorting effect and weaken the ambition of the 2030 Agenda (see also King, 2017).For example, GDP forms the basis of 17 indicators used to measure progress towards 9 goals and 15 targets, when most of these goals and targets do not include GDP in their wording.In the specific case of SDG 10 on reducing inequality, Fukuda-Parr (2019) argues that the indicators have reinterpreted the goal to focus on inclusive growth and poverty, and shifted the debate away from issues of extreme inequality in terms of wealth and income.This is in line with the interests of the World Bank, which was centrally involved in the indicator selection process.
The performativity of indicators is an inherent aspect of their design.To mitigate the risk of unintended consequences, it is necessary to understand the specific features of the current indicator framework that may be causing harm and make targeted improvements accordingly.This requires a deep understanding of the complex dynamics at play and a commitment to ongoing refinement of the indicators.

Goodhart's law: measures tend to become targets
The basic premise underlying the use of any indicator is that statistical correlation exists between a variable that needs to be measured and a variable that can be measured.Importantly, however, this correlation is not stable over time and may be interfered with.
This pitfall is neatly summarized by the economist Charles Goodhart in the following way: 'Any observed statistical regularity will tend to break down when pressure is applied to it for control purposes' (Goodhart, 1975).Goodhart argues that actors often start behaving strategically to appear compliant by simply focusing on what is being measured as a proxy, when they are not actually contributing to making genuine social progress towards the target.A familiar case in point is measuring the performance of researchers with the h-index, which encourages perverse behavior such as unjustified self-citation (Chapman et al., 2019).The so-called Goodhart's law has become better known in its popularized form put forward by the anthropologist Marilyn Strathern (1997: 308): 'When a measure becomes a target, it ceases to be a good measure'.
Preventing measures from becoming targets is crucial for achieving desired outcomes.One important way to achieve this is by ensuring a balance between the number and variety of indicators used in relation to the goals or targets they are designed to serve.Having too few indicators for a specific target can lead to an over-emphasis on those indicators and an increased risk of redefining the target to align with the indicators, rather than the other way around.This can be seen in the over-reliance on the h-index as a key indicator of scientific impact, where the objective of increasing one's index score takes precedence over the goal of increasing overall scientific impact.Similarly, using GDP growth rate as the sole indicator of the health of the economy can lead to the objective of boosting GDP becoming the primary focus, rather than considering the well-being of all actors in the economy.To avoid such issues, it is important to use a diverse set of indicators that are relevant and appropriate for the target being measured, rather than relying on one or two indicators alone.
In the case of the SDGs, the number of indicators, which stands at 231, may appear abundant, especially when compared to the 60 indicators of the MDGs.It should be noted however that the MDG framework had significantly fewer goals and targets, only 8 and 21 respectively.Furthermore, many of the SDG targets are cocktails of issues that cannot be amalgamated into a single measure (MacFeely, 2020).In fact, it has been estimated that between 500 and 650 indicators would be needed to effectively address key aspects of the 169 SDG targets (MacFeely, 2020;Kubiszewski et al., 2021).
Fig. 1 shows that the average number of indicators per SDG target is 1.5, which is significantly fewer compared to the average of 2.9 indicators per target under the MDGs.Only 12 targets have three or more indicators.Strikingly, progress towards 62% (or 105) of 169 SDG targets is tracked using one indicator per target.While a few of these indicators are composite indices (e.g., Indicator 15.5.1 or the Red List Index), most are not.Some goals are more troubling than others; for example, all ten targets of SDG 14 are measured using only one indicator each.The design of such a streamlined indicator framework was a political decision: statisticians were instructed to create 'the framework with the least number of indicators' (Ordaz, 2019: 142), and specifically to limit the number of indicators to one per target (MacFeely, 2020).
The use of a limited number of indicators per target can lead to problems with incentives and measurement.The indicators may prioritize certain aspects of the SDGs over others and fail to capture the full scope of sustainable development.For example, Target 8.9 aims to promote sustainable tourism, but its sole indicator focuses narrowly on increasing the proportion of tourism GDP to total GDP.This narrow focus on GDP can lead to negative consequences, such as neglecting the environmental and social impacts of tourism.Similarly, Target 14.7 aims to increase economic benefits from sustainable use of marine resources, but its sole indicator measures the value added of sustainable marine capture fisheries as a proportion of GDP.This indicator fails to capture important aspects of sustainable marine resources management such as preserving biodiversity, protecting coastal communities, and ensuring sustainable livelihoods.
When such indicators with a narrow scope shape the implementation of the SDGs, the integrity and coherence of the SDG framework may be compromised (Coscieme et al., 2019).This is because, while the wording of the goals and targets are largely complementary (Le Blanc, 2015), significant trade-offs exist at the level of indicators (Pradhan et al., 2017;Barbier and Burgess, 2019).For example, improving the 'annual growth rate of real GDP per capita' (Indicator 8.1.1)may have a negative impact on 'material footprint, material footprint per capita, and material footprint per GDP' (Indicator 12.2.1),and vice versa.Target 17.14 aims to lessen the impact by enhancing policy coherence, but ironically it is also supported by the sole indicator that tracks progress by focussing on whether a country has put in place 'mechanisms' such as inter-ministerial committees.Yunita et al. (2022: 93) argue that this indicator, when used alone, 'risks neglecting how and whether sustainable development … is (trans)formed'.

Campbell's law: numbers are easily corruptible
Preventing indicators from creating perverse incentives is particularly challenging when the measures are quantitative.This is succinctly summarized by the psychologist Donald Campbell in what has come to be popularly known as Campbell's law: 'The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor' (Campbell, 1979: 85).Numbers are easily manipulated, yet they are powerful (Fukuda--Parr et al., 2014).
A classic example of Campbell's law is the McNamara fallacy, named after Robert McNamara, who was the United States Defense Secretary during the Vietnam War.McNamara believed that quantitative measures such as weapons seized, prisoners taken, sorties flown, and body count could accurately gauge the progress of the war (McNamara and VanDeMark, 1995).However, despite these measures indicating otherwise, the war did not end as he had expected.This was due to the corruption of these quantitative measures, with numbers being deliberately inflated by the Unites States army in their operations reports.As a result, military leadership made decisions based on manipulated data, which prevented it from achieving the desired outcome.
Such distorting effects of the numerical approach were observed in the implementation of the MDGs, especially where indicators were poorly chosen (e.g., Fukuda-Parr et al., 2014;Sen and Mukherjee, 2014;Unterhalter, 2014;Yamin and Boulanger, 2014).These studies have shown how numerical measures often involve problematic reification through abstraction and simplification.They also highlight how pervasive and systemic the issue can be.As an example, it has been reported that some local government officials in China manipulate regional GDP numbers to meet national economic growth targets (Lyu et al., 2018).The risk of distortion is inherent in the practice of governance through goals, but it can be aggravated by the selection of quantitative indicators selected based on practical criteria such as data availability.
Yet the SDG indicators remain quantitative, with a few units used most frequently.Out of 231 indicators, 98 measure 'proportion' (e.g., proportion of population living below the poverty line), 30 gauge 'rate' (e.g., participation rate in organized learning), and 30 count 'number' (e.g., number of commercial bank branches).Other units include, for example, 'amount' of fossil-fuel subsidies (Indicator 12.c.1),'coverage' of protected areas (Indicator 14.5.1),'prevalence' of undernourishment (Indicator 2.1.1),passenger and freight 'volumes' (Indicator 9.1.2),health worker 'density' (Indicator 3.c.1),and research and development Fig. 1.The percentage of SDG targets measured using a single indicator (represented by bars, with an average of 62%) and the average number of indicators used per target (represented by markers, with an average of 1.5).For example, the progress towards SDG 6 is measured using 1.4 indicators per target, with 63% of targets having a single indicator.
'expenditure' (Indicator 9.5.1).There are also a few yes or no indicators, such as the 'existence' of certain legal frameworks or national strategies (Indicators 5.1.1 and 8.b.1).
The scores on these quantitative indicators are relatively easier to improve than it is to realize the true meaning of their goals and targets.Take Indicator 12.6.1,for example, which tracks the 'number of companies publishing sustainability reports'.This simple count can be easily improved without actual progress being made.In fact, while the number of reports almost doubled by 2020, many of them did not mention critical aspects of environmental, social, and governance domains (United Nations, 2020).A similar issue was found in a study on Indicator 11.2.1 which measures the 'proportion of population that has convenient access to public transport'.The city of Bogotá managed to improve the indicator score simply by increasing the number of bus stops.This measure made it appear that the target is being met, but in fact it decreased the average speed of buses and resulted in slower service and reduced mobility overall (Brussel et al., 2019).
Reliance on quantifiable measures alone may not only have detrimental effects on measurement accuracy (Lyytimäki et al., 2022), but could also prevent societal transformation.This is because existing data collected by national statistics offices often mirror the entrenched values and vested interests of governments or other powerful actors, rather than those of a sustainable society.For example, using quantitative metrics such as GDP would prolong business as usual and impede a shift towards a steady-state economy (O'Neill, 2012).Like GDP, many indicators used to measure progress towards the SDGs were not specifically designed for that purpose (MacFeely, 2020).Where the development of alternative measures of progress that complement GDP is called for (Target 17.19), no indicator has been adopted.The SDGs should be driving a data revolution, but they have been rather constrained by the data already available (Unterhalter, 2019).

Diversify, connect, and innovate
The foregoing analysis suggests that while there are practical challenges in monitoring progress towards the SDGs, streamlining the indicator framework may lead to unintended consequences.This is because the number of goals and targets is relatively large compared to the number of indicators.An alternative approach I propose is to augment the SDG indicator framework and make it more robust and comprehensive.There are three ways to achieve this: diversification, systematization, and innovation.
First, diversify the indicators used for each target.This would involve adding more indicators to the point where most, if not all, critical aspects of the targets are captured.Indicator diversification would ensure that multiple perspectives are considered, and the focus remains on desired outcomes rather than on the indicators themselves.A good example is Target 10.7, which is supported by four indicators that measure various aspects of migration such as the risks of migration routes, the state of national migration policies, and the number of people and countries involved.This multifaceted approach can be applied to other targets to ensure a comprehensive and holistic view of progress.Where useful, the overall progress on each target could be monitored by applying a generalized mean of individual indicators (Rickels et al., 2016).
The use of a diverse set of indicators for monitoring progress may come with an increase in the cost of monitoring, which can be particularly challenging for resource-constrained, least-developed countries.This challenge is often used as a justification for the mainstream view: using fewer, but more relevant indicators.However, this approach is not sustainable.A limited set of indicators would increase the risk of corruption and the need for regular replacement.A more long-term solution is to prioritize investing in building statistical capacity in developing countries through training, resources, and support (Target 17.19).This will not only help them to meet their monitoring needs, but also contribute to their overall development.
Second, use indicators to connect targets within and between the SDGs.One possibility is to repeat and use indicators for multiple targets, linking several issues through these indicators (e.g., Dora et al., 2015).This approach addresses the extra burden on data management that may arise from increasing the number of indicators.Currently, only thirteen indicators are repeated, with nine of them spanning across multiple goals.An example is the 'number of deaths, missing persons and directly affected persons attributed to disasters per 100,000 population', which is used for measuring progress on Targets 1.5, 11.5, and 13.1.Other indicators could follow the same approach, thus mitigating the distorting effects of sole indicators, addressing trade-offs, and improving coherence between the SDGs.
Repeating indicators can make it possible to assign three indicators per target to address the social, economic, and environmental dimensions of sustainable development (e.g., Lyytimäki et al., 2020).This approach addresses the silos between the SDGs, which have left the SDGs vulnerable to cherry-picking or prioritization (Forestier and Kim, 2020).It is similar to the proposal made during the SDG negotiations, which called for every SDG to be supported by three categories of social, integrated, and biophysical targets (Griggs et al., 2014).
Third, develop qualitative indicators of social progress.Such indicators can provide qualitative information for missing quantitative data points or help interpret quantitative measures.Examples include measuring the impact of tourism on local culture through resident surveys and evaluating the depth and transparency of corporate sustainability reports through stakeholder engagement.Developing qualitative indicators may require innovation.But once developed, they can complement traditional quantitative measures that may not fully capture the complexity and nuances of sustainable development.
A concrete example of developing alternative indicators is the creation of a new measure that could complement and eventually replace GDP (Kubiszewski et al., 2013;Costanza et al., 2014).The SDG framework, specifically Target 17.19, already provides a basis for organizing such an effort.This target highlights the need to move beyond indicators such as GDP and to embrace 'well-being', 'happiness', or 'life satisfaction' as key measures (Costanza et al., 2016;Fioramonti et al., 2019).The methodological challenge of measuring such less quantifiable, often subjective, context-specific, and self-reported indicators is real, but there have been successful attempts such as Bhutan's use of gross national happiness (Ura and Galay, 2004;Brooks, 2013).Other examples include the Better Life Index (Mizobuchi, 2014) and Social Progress Index (Porter et al., 2013).These examples serve as a reminder that the challenge of going beyond GDP is not technical or financial, but rather political.

Conclusion
Indicators are not just tools for measurement; they are performative.They redefine goals and targets, allocate resources, and influence behavior.As some feared, the indicator framework has been used as 'a back door to changing the SDGs and targets' (Kamau et al., 2018: 224).Seen from this light, streamlining the current set of SDG indicators as some suggest may further weaken the ambition, integrity, and legitimacy of the SDG framework.The indicators will need to be rather augmented.
Effective governance through goals necessitates a shared understanding of the nature of indicators and their purpose.Indicators are boundary objects, and they should be crafted accordingly by taking into account the perspectives of all relevant stakeholders.Monitoring the SDGs is not just a statistical undertaking; it is an iterative process of dialog at the science-policy interface where questions like what should be measured, why it should be measured, and by whom are continuously debated (Pintér et al., 2017;Kanie, 2020).It is questionable however whether the IAEG-SDGs has effectively functioned as a boundary organization in this process.
As we look to the future development agenda beyond 2030, we need to start discussing how to design a new goal framework that is informed by the considerations of indicators.I argue that it is imperative to incorporate indicator development as an integral part of the negotiation of post-2030 global goals.This would mean that statisticians would engage in goal and target setting from the outset to share their perspectives and inform decision-makers of any methodological challenges.Decision-makers would also actively participate in indicator development to ensure that goals and targets accurately convey their intended meanings.It was a deliberate political decision of the Open Working Group on Sustainable Development Goals to 'leave the indicators to national statistical bodies', rather than to develop the indicators themselves in conjunction with the goals and targets (Kamau et al., 2018: 144).A different choice can be made.The data revolution is a prerequisite for sustainability transformation (Espey, 2019).It is not just about having data, but also about the type of data collected to measure genuine progress.To go beyond the status quo, we must consider options beyond cost-effectiveness and expand the horizon of possibilities.This includes prioritizing data and transferring resources from developed to developing countries.Ultimately, the SDGs will be transformative to the extent their indicators allow.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
February 2023 performativity of indicators, or ability to shape outcomes.By drawing on the literature on conditions under which indicators have unintended consequences, I argue that there are rather limited, exclusively quantitative indicators in the 2030 Agenda to adequately address the multifaceted aspects of diverse targets they serve.In fact, each target is typically measured with only 1.5 indicators, which monitor quantifiable changes in proportion, rate, amount, and the like.A staggering 62% of the targets are supported by sole indicators, effectively equating progress measured on the 105 indicators with progress on the 105 targets.