A Momentum for Change? Systemic effects and catalytic impacts of transnational climate action

Non-state and subnational climate actors increasingly commit to act across borders to reduce greenhouse gas emissions, to help communities adapt to climate change, and to push governments into more ambitious climate policies. The effectiveness of such transnational climate initiatives, however, is still largely unknown. Current studies often only seek to estimate the mitigation potential of such initiatives or to study the design elements that may be more or less conducive to their effectiveness. Little is known about the impacts of such initiatives on broader social and environmental goals and about their “ catalytic ” impacts, that is, whether such transnational initiatives can grow and possibly replicate. Here we develop an approach inspired by political systems theory to reach a fuller understanding of the effectiveness of transnational initiatives. We operationalize a generalized framework through a combination of methodologies, using a new dataset of climate actions under the Mo-mentum for Change program of the United Nations Framework Convention on Climate Change, which we combine with surveys, database analysis, and contextualizing interviews. We conclude with a reflection on the applicability of the framework and a discussion on opportunities for the Momentum for Change program to strengthen its efforts.


Introduction
The effectiveness of global and regional institutions has always been a central concern in scholarship on earth system governance.However, assessing the effectiveness of transboundary institutions remains challenging both conceptually and empirically.In the study of intergovernmental treaty regimes, 1 many approaches have been explored to measure effectiveness, including the tracking of governmental compliance, behavioral effects, and effects against real or hypothetical counterfactuals (Hovi et al., 2003;Mitchell, 2006;Underdal, 2010).Separate assessments that use different methodologies, however, often do not necessarily add up to a more coherent understanding of effectiveness, even when the same regime is studied.
These problems are even more complex when it comes to institutions beyond intergovernmental cooperation, including the many political institutions that have been set up by non-state actors, such as environmentalist organizations or corporations, or by subnational public authorities, such as cities.We call such institutions, in line with common usage in the literature, "transnational" governance arrangements, which we define here as institutions that seek to collaborate towards shared public policy objectives across national boundaries and that include at least one non-state or subnational actor (such as businesses, civil society organizations or cities).The assessment of the effectiveness of such transnational governance arrangements remains challenging for numerous reasons.First, most data are collected at the scale of national jurisdictions and cannot be easily disaggregated to measure the impact of transnational programs.Conversely, when effects of individual transnational initiatives are known, they cannot simply be aggregated because national datasets might already include such effects, leading then to double counting (Bansard et al., 2017;Bakhtiari, 2018).Moreover, the voluntariness and individualized designs of commitments by transnational actors complicate comparative research.Compared to intergovernmental regimes, the attribution of observed effects in transnational governance is an even greater challenge.The sheer number of transnational initiatives obscures any distinction between their initial commitments to action and their eventual observable activities.
Despite these challenges, numerous studies have recently investigated the impacts of transnational initiatives, especially in climate governance.For example, Michaelowa and Michaelowa (2017) have assessed transnational climate actions by their design features in light of how they could contribute to their effectiveness.Most studies on the effectiveness of transnational climate initiatives, however, focus on their potential as expressed in quantitative terms, particularly in closing the "global mitigation gap" (e.g., Blok et al., 2012;den Elzen et al., 2011;Hsu et al., 2015Hsu et al., , 2016;;Graichen et al., 2017;Kuramochi et al., 2020;Lui et al., 2020;Roelfsema et al., 2018;UNEP, 2016UNEP, , 2017UNEP, , 2018)).This focus on the mitigation potential has raised some concerns.For instance, it may lead to overly optimistic estimations of the effects of climate action.Moreover, because the full mitigation potential is rarely met, ex post assessments remain necessary (Chan et al., 2019).Prompted by these concerns, a few scholars have advanced other methods that go beyond the pure promises of transnational climate initiatives.For instance, consecutive editions of the Yearbook of Global Climate Action (2017a;2018;2019) as well as Chan et al. (2018Chan et al. ( , 2019) ) have emphasized the need to carefully identify any relevant and attributable outputs of transnational climate actions as the necessary preconditions for their later effectiveness.These studies argue that only those transnational climate actions that have specific and relevant outputs are likely to have any later impacts.Conversely, this focus on outputs as a precondition to effectiveness does not allow to assess the eventual magnitude of effects, for instance in terms of number of people affected, amounts of funding, or scale of emissions reductions.Some first steps have therefore been taken to combine ex-post and ex-ante assessment methods.NewClimate et al. (2019), for instance, uses output-based assessments in the sampling of climate initiatives before their mitigation potential is assessed.And yet, even such combination and sequencing of methods leaves many research gaps.Notably, the assessment of the samples themselves remains mostly based on self-declared targets, and their impact in terms of changes in social and environmental indicators largely unknown.
In addition, several scholars have emphasized that the indirect impacts of transnational climate actions may even be more important than their directly attributable impacts.For example, through broader demonstration effects, transnational initiatives can grow in scale and scope, once their proposed solutions are more widely applied (Hoffmann, 2011;Bernstein and Hoffmann, 2018;Bulkeley et al., 2018;Hale, 2018;Chan et al., 2019;Hsu et al., 2019).Such catalytic indirect impacts, however, are not well understood (Hale, 2016(Hale, , 2018)), and current analyses of transnational initiatives have not yet considered such impacts systematically.
This paper seeks to drive this debate further.We respond here to the urgent need to interrogate the broader effectiveness of transnational initiatives, in an analysis that goes beyond their potential, beyond their design features that might indicate effectiveness, and beyond their mere production of output as opposed to real impact.Moreover, we consider both the direct inputs and outputs of initiatives, as well as their catalytic impacts, from the expansion of an initiative's activities to the replication of these activities by others.
The paper is organized as follows.In the next section, we present the conceptual design of our work and explain our empirical focus, the Momentum for Change initiative that is operated by the secretariat of the United Nations Framework Convention on Climate Change.Section 3 describes the Momentum for Change program as an experiment in systemic and catalytic change and presents our empirical data analysis.
Section 4 reflects on the application of our analytical framework and discusses the role of the climate convention and opportunities to improve the orchestration of catalytic climate action.

Conceptual framework
We advance and test in this paper an analytical framework to measure the systemic effects of transnational climate action.We draw on Eastonian political systems theory (Easton, 1953(Easton, , 1965) ) in assuming a ladder of policy effects to indicate progress across a policy cycle, that is, a logical progression from inputs, outputs, outcomes and impacts, in which the preceding type of effect is necessary for the next.In our framework, inputs refers to the resources of transnational actions to operationalize their commitments; outputs to observable, tangible, and attributable production; outcomes to behavioral change; and impacts to changes in environmental and social indicators (see also Young, 2011;Hale et al., 2021).Moreover, we try to understand catalytic and amplification effects by assessing whether individual transnational initiatives grow and are replicated across contexts.Consequently, our general analytical model assumes a ladder of policy effects that includes a progression of assessments; from inputs to output performance, to environmental and social outcomes and impacts, to catalytic impacts (Fig. 1).
We use this analytical framework here to study a defined set of transnational initiatives, that is, those initiatives that have been included under the "Momentum for Change" initiative that is administered by the secretariat of the United Nations Framework Convention on Climate Change (UNFCCC).We investigate the extent to which transnational climate initiatives under the Momentum for Change initiative have been effective over time, by looking into inputs in terms of targetsetting and measures to monitor and report implementation; output in terms of whether their attributable and tangible production are consistent with their functions; and catalytic impacts, by considering whether initiatives and their approaches have been applied at a greater scale or been replicated.Unlike other outreach and engagement efforts by the climate convention, the Momentum for Change initiative has attracted only limited scholarly attention.For instance, scholars have noted the initiative as part of a "conceptual shift" within the climate convention secretariat to engage non-state and local actors beyond the statecentered climate regime (Hale 2016: 15; see also : Hickmann and Elsässer, 2020).Other studies have also identified the climate convention secretariat's role in building momentum towards climate action, for instance in climate education (Aykut et al., 2020;Kolleck et al., 2017).However, no attempt has yet been made to assess the Momentum for Change program and the initiatives it awards, even though this program is one of the earliest efforts by the climate convention secretariat to engage nonstate actors and to stimulate their contributions to the climate challenge.Moreover, the Momentum for Change initiative uniquely embeds climate action in broader goals of sustainable development, for instance in its thematic foci on the "Urban Poor" and "Women for Results."Importantly, the secretariat of the climate convention pursues here a catalytic theory of change that seeks to recognize and further strengthen the visibility of the well-performing transnational climate initiatives, hence implying a progression from transnational commitments to broader system-level impacts.
We studied the climate actions under the Momentum for Change initiative using both survey data and self-collected data from publicly available sources such as websites and social media.First, we constructed a database on the basis of a survey (see supplemental information) that generated data on organizational features relating to input effectiveness; on functions and outputs; and on targets and achievements for outcomes and impacts.We sent surveys to all 445 climate initiatives that were either nominees or winners of the "Momentum for Change Lighthouse Awards," which is part of the Momentum for Change program and was featured on the climate convention website until the end of 2016 (see http://unfccc.int/secretariat/momentum_for_change/i tems/7176.php).We received 51 responses between 6 June and July 9, 2016, equaling a response rate of 12%.The respondents include 11 Lighthouse Activity laureates and 40 initiatives that did not win an award but meet the minimal criteria to be nominated according to the climate convention secretariat and expert panel (for criteria, see below).To understand catalytic impacts until October 2019, we additionally collected data on the growth and replication of these initiatives and their solutions, across and within countries.We subsequently compared laureates and non-laureates whether they generate catalytic impacts.
Although the current study provides a first and uniquely comprehensive insight into multiple effects of transnational governance initiatives, findings should be carefully interpreted, with a consideration of possible biases.For instance, although we consider the response rate sufficient for our initial investigation, as the 51 initiatives that responded to our survey represent a wide range of organizational types and thematic foci, caution is necessary when generalizing findings for the whole Momentum for Change initiative.For instance, initiatives that have more activities may also be more likely to respond to surveys, leading to a bias towards more visible and possibly more effective initiatives.In other words, ineffective or largely inactive initiatives are unlikely to respond to a survey and are hence also not covered in a study such as ours.A reliance on self-assessments through surveys is also likely to result in biases through tactical under-or overstatement of achievements.In the assessment of catalytic impacts, such self-assessments are partly triangulated through separate data collection on publicly available resources.However, such data collection could miss out on instances of expansion and replication of activities that are not recorded, e. g., on social media or in online reports.As award-winners may be more closely scrutinized, more data may be available on their catalytic impacts, whereas the growth and replication of non-laureates' activities may be underreported.

Analysis
Taking these caveats into account, we now analyze in detail one set of transnational governance initiatives in climate governance, the Momentum for Change program.We first introduce this program briefly and explain our data selection, followed by our data analysis.

The Momentum for Change program
The Momentum for Change program is an initiative operated by the secretariat of the climate convention to identify and reward innovative and transformative solutions that tackle problems related to climate change as well as wider economic, environmental and social problems.In the words of Patricia Espinosa, "climate change can't only be a topdown process.In the end, it will take citizens, communities, cities, The Momentum for Change initiative is one of the earliest attempts under the climate convention to engage actors beyond governments.It has also been the first example of the climate convention partnering with private actors in programmatic outreach and engagement of non-state and subnational actors.Through partnerships, the Momentum for Change program tries to leverage capacities and resources beyond the contributions by governments.Examples are financial sponsorships of specific themes, the aligning and leveraging of nonmaterial capacities through partnerships and networks, and the occasional engagement of celebrities, for instance in the production of promotional audiovisual communications (Marchildon, 2018).Thematic foci under the Momentum for Change initiatives are "Urban Poor" (until 2015, sponsored by the Bill & Melinda Gates Foundation); "Financing for Climate Friendly Investment" (in partnership with the World Economic Forum Global Project on Climate Change); "Women for Results" (sponsored by Masdar, a United Arab Emirates owned renewable energy and sustainable urban development company); "Planetary Health" (sponsored by the Rockefeller Foundation); "ICT Solutions" (until 2017, in partnership with the Global e-Sustainability Initiative); and "Climate Neutral Now" (an initiative launched in 2015 and led by the climate convention secretariat to encourage all sectors of society to commit to actions to achieve mid-century climate neutrality).
Its wide thematic scope and strong emphasis on sustainable development make the Momentum for Change program more comprehensive compared to other outreach, mobilization and orchestration efforts that the climate convention secretariat has pursued.For instance, the "international cooperative initiatives" (Widerberg and Pattberg, 2015) and most transnational actions in the Non-State Actor Zone for Climate Action administered by the climate convention secretariat focus more narrowly on the mitigation potential (e.g., Roelfsema et al., 2018;Hsu et al., 2016).Moreover, the Momentum for Change initiative combines a multi-thematic focus with a focus on different and often underrepresented stakeholders in climate governance, in particular women in the Global South and the urban poor.The Momentum for Change initiative emphasizes sustainable development broadly and requires all its initiatives to make their specific area of sustainable development explicit, instead of featuring sustainability merely as a side benefit to climate actions.
The establishment of the Momentum for Change initiative in 2011 was motivated by the disappointing outcome of the 2009 conference of the parties to the climate convention in Copenhagen (Davila, 2016).By raising awareness of the diversity of transnational climate action, the climate convention secretariat hoped to change the momentum towards more optimism and more ambition regarding future international climate negotiations.The Momentum for Change initiative also emphasizes the influence of climate actions as it seeks to "shine light into the enormous groundswell of activities underway across the globe that are moving the world toward a highly resilient, low-carbon future" (UNFCCC, 2019).Through a competitive application and selection procedure, the Momentum for Change initiative designates some of initiatives as "Lighthouse Activities," which it defines as "some of the most practical, scalable and replicable examples of what people, businesses, governments and industries are doing to tackle climate change" (UNFCCC, 2020).All eligible climate actions included in the Momentum for Change activity database (until 2016) had to fulfil six criteria.They had: 1. To address climate change mitigation or adaptation; 2. To be in the implementation phase; 3. To be scalable and/or replicable, referring to a foreseeable expansion of impact and geographic spread; 4. To be innovative, which is stated as a call for new business models, technologies, processes or financing structures and transformative, asking for non-incremental, but long-term oriented structural changes; 5. To deliver verifiable social and environmental benefits.6.To not be registered, and not intending to do so, as a clean development mechanism or joint implementation initiative in the next two years.
Particularly the need for eligible climate action to be scalable and replicable (criterion 6) strongly relates with the objective of the current study to understand catalytic impacts.
At the time of our data collection, 48 out of 445 eligible climate actions featured in the Momentum for Change Activities Database in 2011-2015 had been recognized as such "Lighthouse Activities."Since then, the number of applications has increased dramatically.For instance, in 2017 the Momentum for Change program received 462 applications, 223 of which were considered eligible.Yet only 19 were designated as a Lighthouse Activity.
The Momentum for Change Activities Database has been discontinued in 2016 due to a lack of resources, and nonwinning activities are no longer featured.Those initiatives that are recognized as a Lighthouse Activity, however, continue to be featured on the website of the climate convention.While lack of resources has restricted the operation of the Momentum for Change initiative, the theory of change underlying it does not require large resources; it relies instead on the power of visibility and assumed proliferation of actions due to scaling and replication.Chan et al. (2019) describe this as a "more brings more" motivation.By raising the visibility of these few successful examples of climate action, the program acts on the assumption that these best-practice actions will grow and replicate.As a result, Momentum for Change includes relatively few initiatives, while other mobilization efforts under the climate convention, such as the Non-State Actor Zone for Climate Action, are much less discriminate.The annual selection of 19 lighthouse activities (in 2018) is a trifle compared to the more than 29, 000 climate actions featured on the platform Non-State Actor Zone for Climate Action (in June 2021).The expected catalytic course of change is not easily realized, however.For instance, Momentum for Change's catalytic theory of change assumes that "best practices" can be effectively identified; that their visibility will attract sufficient attention of other stakeholders, including investors, journalists, and peers; and that greater attentional and networking will ultimately lead to a broader uptake of "best practice" approaches by others and across different contexts.Hence, the aim to select "outstanding" initiatives raises questions about how representative this process is and whether the people involved are able to judge based on the limited reports of applicants.
The Momentum for Change project team, which is part of the climate convention secretariat, is central to this engagement effort.It has prepared annual calls and undertakes large-scale communication efforts.It has also organized events, especially at climate conferences, to feature the winning initiatives.The most important procedure, however, is the selection of Lighthouse Activity laureates.The selection is organized in three steps.Following a call for applications, the climate convention secretariat reviews all applications for their eligibility.Eligible applications are then sent to an advisory panel.The advisory panel, currently 23 experts from different disciplinary and geographic backgrounds, is asked to take into account the six above-mentioned criteria, along with more specific criteria related to the thematic focus and the additional objective to have a broad geographic representation of laureates.The selected Lighthouse Activities receive a package aimed at increasing their visibility to a larger audience and potential partners and investors, along with fully subsidized attendance at the climate convention conference.However, most benefits are in-kind and include access to policymakers and potential funders during the conference; recognition by the climate convention secretariat; public relations support; media training; marketing materials and promotional videos; professional photography; and a dedicated page about the winning initiative on the climate convention website.
The effectiveness of the Momentum for Change initiative hinges on whether it captures the most outstanding climate actions.Although the selection procedures request the applicants to show that their plans are also implemented, relying on their self-reported data remains risky.Applicants can easily overstate or tactically understate achievements to outshine competitors or to attract additional resources.Moreover, any broader, catalytic impacts are far from guaranteed.For instance, the exposure to policymakers may be quite limited when engagements coincide with busy international negotiations.In the following, we now try to understand different types of effectiveness.

Sample characteristics
Our dataset differs from earlier large-n analyses of transnational climate actions in four ways.First, our dataset has as lead partners more actors from industry and business, including small and medium-sized businesses, and non-profit/non-governmental organizations.Previous studies, in contrast, had as lead partners more often the traditional actors such as international organizations and governments (Bulkeley et al., 2012;Chan et al., 2015Chan et al., , 2018;;Pattberg et al., 2012).Second, our dataset does not show the typical overrepresentation of mitigation actions; instead, there is a stronger focus on adaptation, with most initiatives in our sample focusing on adaptation benefits or aim at mitigation and adaptation benefits equally.Third, in terms of governance functions, our dataset has an uncharacteristically large share of transnational actions aiming at technical, on-the-ground implementation (28%), while previous studies recorded very few of such actions, having a functional focus more on knowledge dissemination, helping (public) policy planning, and knowledge production and dissemination (Chan et al., 2015;Chan and Amling, 2019;Pattberg et al., 2012).Fourth, our dataset has a strong participation of, and implementation in, developing countries, whereas previous studies show a strong overrepresentation of Europe-based and North America-based participation andto a lesser extentimplementation.In fact, most lead partners in our data set are based in Africa (16 cases), followed by Europe (11), Asia (9) and South America (7).We also find that most implementation takes place in developing countries (respectively 26% in low-income countries, 40% in lower-middle income countries, and 24% in upper middle-income countries).
These five descriptive features show that the Momentum for Change program includes a distinct set of transnational climate actions that seems to contradict patterns found in other studies.To some extent, the Momentum for Change program might even remedy some widely observed flaws of transnational climate action and contradict the "Northern bias" in participatory patterns that is often found in transnational governance studies.The Momentum for Change initiative also demonstrates strong and frequent leadership by private actors, including small and medium enterprises and local civil society; and suggests that a large share of benefits accrues in the Global South.
However, does this rather uncharacteristic sample of initiatives also deliver?

Input effectiveness
We start our analysis with looking at input effectiveness, that is, the procedural and organizational robustness of the transnational climate initiatives themselves (Underdal, 2004;Michaelowa and Michaelowa, 2017).We particularly studied the institutional quality and capacity of initiatives, which we operationalized by two assessment criteria with eight variables: first, institutional quality (operationalized by monitoring arrangements; reporting arrangements; evaluation arrangements; baseline scenario; and quantitative targets); and second, institutional capacity (operationalized by dedicated staff; task division; and steering organ).
Using the two criteria, we find quite robust institutional arrangements.95% of all 51 initiatives in our dataset declare to have monitoring arrangements (Fig. 2), and 76% regularly report on their activities.However, few made their baseline scenarios explicit, which complicates later evaluation of reported activities and achievements.
In terms of institutional capacity, we also see a robust picture emerging: 84% of the initiatives clearly divide tasks within their organization, and most initiatives have central coordination mechanisms, such as a secretariat (52%), or networked coordination, such as meetings of representatives of partner organizations in steering groups and advisory panels (34%).Based on our indicators for institutional robustness and compared to previous studies (e.g., Michaelowa and Michaelowa, 2017), we conclude that the large majority of the 51 initiatives in our sample meet at least some minimal requirements in terms of institutional capacity.We could not, however, evaluate further indicators, such as the quality of coordination mechanisms, that is, whether monitoring mechanisms result in learning, or the quality and level of ambition of targets.Therefore, seemingly robust institutional arrangements and capacities in our analysis by no means guarantee substantial efforts.

Output performance
To assess output performance, we use the so-called Function-Output-Fit method that we developed earlier (Chan, 2009;Pattberg et al., 2012).The Function-Output-Fit method has been applied to influential samples of transnational initiatives, including "private-public partnerships for sustainable development" (Chan, 2009;Pattberg et al., 2012) and transnational climate actions; e.g., those emerging from the 2014 New York Summit (Chan et al., 2018); and larger sets of UN-registered transnational initiatives (UNFCCC, 2017;2018;Chan and Amling 2019).The Function-Output-Fit method links data on tangible and attributable outputs with data on 12 inductively defined governance function categories2 to assess the extent to which outputs correspond with the underlying functions.An initiative can score a "partial Function-Output-Fit" if it produces fitting output(s) for at least one relevant function.When an initiative has outputs for all relevant functions, we consider it having a "full Function-Output-Fit."By contrast, "no Function-Output-Fit" refers to initiatives that do not produce any relevant outputs.
Our overall findings show that all 51 initiatives had produced some output, and fewer 20% lacked any outputs relevant to their main functions.37% of the initiatives score a full Function-Output-Fit (Fig. 3).43% of the initiatives have at least some function-relevant outputs.Separate findings for Lighthouse laureates and non-laureates show considerably higher output performance among laureates.
The output performance of our set of 51 Momentum for Change initiatives compares favorably to earlier studies (Fig. 4).For instance, over 300 "Partnerships for Sustainable Development" that were launched at the 2002 World Summit on Sustainable Development (Pattberg et al., 2012) still performed poorly ten years after their launch, and although climate actions that emerged from the 2014 UN Climate Summit in New York (Chan et al., 2018) fared better, the 51 Momentum for Change initiatives in our sample still perform better.
The favorable findings on the performance of Momentum for Change initiatives relate of course to the specific selective process.After all, the program nominates and awards only best practices and proven solutions.Nonetheless, our findings also suggest that the selection procedure is effective in identifying best practices and solutions even though it is based on self-reported application data, a minimal requirement within the change theory underlying the Momentum for Change initiative, namely further scaling and replication through enhanced visibility of outstanding initiatives.

Social and environmental outcomes and impacts
We now discuss our findings regarding the broader social and environmental impacts of the 51 Momentum for Change initiatives that we studied.We looked into five types of targets that the initiatives frequently set: the number of people, villages, cities and countries that they reach, and avoided emissions in terms of carbon dioxide equivalents.We compared these targets with data from the survey responses on the actual achievements towards these target indicators.To accommodate for different types of measures and measurements, we clustered statements on achievements from our survey by percentages of target ambitions, distinguishing four values: achievements below 50% of the original set of targets (Score 1); between 50 and 99,9% (Score 2); 100% (Score 3); and above 100% (Score 4) (Fig. 5).
Even though 41 initiatives had quantitative targets, only 23 of those provided quantitative data on their ambitions and achievements that are compatible to our target categories.Many individualized targets may hence be still difficult to comprehensively categorize by our categories.Moreover, among the 51 initiatives that we studied, the balance between laureates and non-laureates was highly skewed towards the latter: 18 non-laureates provided quantitative indications on ambitions and achievements, compared to only 5 laureates.The lack of data and the stark imbalance in representation between the two types of initiatives in our data makes a valid comparison problematic.Therefore, we do not break our findings down for laureates and non-laureates.
We found 20 targets related to the number of countries they reached, 17 on cities, 15 on villages, 23 on people, and 13 on amounts of avoided emissions.For all targets we found a few over-performers (>100%), and a sizable number of initiatives indicated that they completely fulfill their targets (±100%).Targets expressed in numbers of countries reached have most frequently been achieved.However, emissions targets have most frequently not been reached compared to stated targets (below 50%), although we also saw instances where these targets were more than reached.
Overall, we do not see a clear trend in terms of the relative impact of initiatives by target category, let alone their environmental and social impacts in absolute terms. 3Moreover, even if these initiatives make substantial impacts, they not necessarily add to those of other actorsfor instance governments -(see e.g., Hsu et al., 2019).They may even substitute potentially more impactful action by other actors.From Momentum for Change's catalytic change theory, however, the most important effect of transnational initiatives in climate governance is whether initiatives, however small, would grow and be replicated.We discuss this next.

Catalytic impacts
Assessing catalytic impacts is still fraught with difficulties.
Nonetheless, we tried to use our data to better understand (1) patterns of growth and replication in our sample by looking at whether an initiative has significantly scaled its activities from their launch until October 2019; and whether (2) Lighthouse laureates have grown and replicated more compared to non-laureates.
First, we compared the number of countries in which initiatives were first implemented with the number of countries in which they were eventually implemented (by October 2019).We considered replication    S. Chan et al. across scales of governance and the types of growth since their awarding.As for growth, we distinguished seven types of growth: more emissions reduction; more energy efficiency; more renewable energy; more people positively impacted; more areas protected; more market share and sales; and more funds mobilized.Concerning replication, we first tried to operationalize this as the reproduction of an initiative's activities across different scales of governance either by the initiative itself or by others.However, we could not identify whether activities were replicated by the initial initiative or by others due to a lack of data and specification in self-reporting.In the end, therefore, our understanding of replication overlaps with our understanding of growth, and we distinguish multiple scales of replication, namely within a country, within a world region (we distinguish 11 world regions, based on UN classifications), and across world regions.
We identified most growth in terms of number of people positively impacted, particularly among by non-laureate initiatives, as well as significant growth in the numbers of organizations engaged among laureate initiatives (Fig. 6).We found less growth, however, in those categories that are more often associated with climate mitigation, notably emissions reduction but also increases in energy efficiency and deployment of renewable energy.This might be linked to the broader focus of the Momentum for Change program on sustainable development, resilience and climate adaptation, compared to other initiatives to engage transnational actors under the climate convention.
In terms of replication across scales, by October 2019 about 62% of the initiatives had grown in the countries of their implementation since their launch, and in the case of laureate initiatives, since the moment of their award.However, replication across countries in the same region (16%) and worldwide (16%) occurred at a much lower rate.These findings show that growth and replication occur more often in the original country contexts in which the initiatives initially operated.Replication also seems to occur more often within than across regions.However, there are remarkable differences in replication across scales between Momentum for Change laureates and non-laureates.Laureates replicate more at all scales of governance, grow more within countries, and are more often replicated within regions and across regions (Fig. 7).
The magnitude of scaling is consistent with expectations according to Momentum for Change's theory of change, and the Lighthouse Activity award seems to coincide with more replication.However, causal effects cannot be definitively attributed.Theories that emphasize network effects suggest that the exposure to other partners and resourcesfacilitated by an initiative such as the Momentum for Change initiativewould indeed contribute to effective growth and replication (Börzel, 1998).Results may show that the Momentum for Change initiative effectively selects initiatives that hold the greatest promise for replication and growth.However, most observed growth and replication may have also occurred in absence of the Momentum for Change program, and our current study cannot account for that possibility.

Conclusion
Against the background of the 2015 Paris Climate Agreement and its strong emphasis on encouraging non-state climate action (see Chan et al., 2016), it is vital to develop and test more sophisticated methods to analyze the potential catalytic impacts of such transnational climate initiatives.Our study has sought to contribute to this important challenge as a first pioneering attempt, even though more research is urgently needed.
Empirically, our study tried to advance understanding on the effectiveness of the Momentum for Change initiative operated by the climate convention secretariat, and especially of those climate actions that were recognized as Momentum for Change "Lighthouse Activities."The Momentum for Change initiative stands out among a growing number of international efforts to recognize transnational climate initiatives (Chan et al., 2018;Hale and Roger, 2014;Bäckstrand and Kuyper, 2017).The initiative is one of the earlier of such efforts, and unusual is several respects: it focuses more broadly on sustainable development than other initiatives; it features a high number of initiatives in developing countries; and it pioneered partnerships with thematic sponsors.Where previous studies of transnational climate initiatives find a strong overrepresentation of (mostly global North-based) mitigation-focused climate action and underperformance of adaptation and resilience focused actions, the Momentum for Change initiative might respond to some of these imbalances.
Although comprehensive in considering both input, output, impact and catalytic impacts, our analytical framework to investigate effectiveness is also limited; each type of the effects that we studied merits further investigation.First, while most initiatives meet our minimal input effectiveness criteria (institutional capacity and quality), most do not report baseline scenarios, which complicates subsequent assessments of impacts in absolute terms.Second, a higher output performance compared to other sets of transnational governance initiatives is hardly surprising because the Momentum for Change program only nominates proven activities.Considering this selection bias, one can consider it as disappointing that only 37% of these initiatives have a high output performance (that is, functions that are matched by relevant outputs).Third, further data collection, among others on baseline scenarios, is needed to determine impacts in absolute terms.However, even our cursory analysis shows that in most cases originally stated targets are not fully achieved, particularly achievements compared to mitigation targets are poor.Finally, even if the assessment of catalytic impacts is complicated for instance by overlapping understandings of growth  and replication, and activities by non-laureates may be underreported, we clearly discern more catalytic impacts among Momentum for Change laureate initiatives.Our findings are consistent with, but do not verify, the assumption underlying the Momentum of Change initiative that increased visibility through such awards can help generate catalytic impacts, as laureates grow and show higher rates of replication at all scales of governance.
Our interrogation shows the viability of comprehensively assessing inputs, outputs, and impacts of transnational governance initiatives, as well as its limitations.For instance, while our analytical framework relates input, output, impact and catalytic impacts, our subsequent stepwise and aggregate analysis cannot explain causality.In this regard, our methodological aim to show the applicability of a political systemsbased framework that sequentially applies different methods to assess aspects of effectiveness is only partly met.For instance, our elaboration of the conceptual framework assumes the more or less equal weight of different dimensions of effectiveness; whereasarguablyproblem solving relates most to impacts.In this regard, we consider inputs and outputs as necessary but insufficient elements in a causal chain from initial intentions of an initiative towards tangible results, impacts, and subsequent growth and replication substantive impacts (see also : Hale et al., 2021).Seen against such a framework of causal progress, then, the fact that we do not find better performance on inputs and outputs may be a matter of concernparticularly among a set of initiatives that has either been nominated for, or won, an award.
While our combination of aggregate assessments gives first indications for the effectiveness of large number of transnational climate initiatives, more empirical research is needed to study causality in a process from the intentions of an initiative to tangible results.Not in the least, in-depth case studies are needed to understand causalities between different types of effects, and to determine the effects of recognizing and rewarding "best-practice" initiatives.Assessments of catalytic impacts of larger sets of transnational initiatives, moreover, would benefit from better data on growth and replication of transnational initiatives.Currently, neither the Momentum for Change initiative nor other climate action platforms such as the Non-state Actor Zone for Climate Action platform or the UNEP-DTU Partnership's Climate Initiatives Platform provide systematic information on for instance the expansion of activities transnational governance initiatives into new areas, their growth in membership or new coalitions they create.Given the strong assumptions underlying the Momentum for Change program about replication and growth, more efforts are hence needed towards collecting data, particularly after initiatives have been awarded.

Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
states and businesses to transform high-level decisions into real-world action.That is what Momentum for Change is all about.It publicizes some of the most innovative, scalable and practical examples of what people across the globe are doing to combat climate change.We call these examples Lighthouse Activities […] real-world reminders that climate action is not just possible − it's the path we must get on [ …. ] (UNFCCC 2017b)."

Fig. 2 .
Fig. 2. Score of initiatives on the single quality criteria.

Fig. 7 .
Fig. 7. Growth and replication by laureates and non-laureates within and across countries and regions.

Sander Chan :
Conceptualization, formulation of ideas, overall research goals and aims, Methodology, development, design, Validation, verification of results, Investigation, conducting the research, Data curation, data collation, Writingoriginal and review drafts, Visualization, Supervision, Project administration.Friederike Eichhorn: Conceptualization, formulation of ideas, overall research goals and aims, Methodology, development, design, Formal analysis, Investigation, conducting the research, Data curation, data collation, Writingoriginal and review drafts.Frank Biermann: Validation, verification of results, Writingoriginal and review drafts, Supervision.Aron Teunissen: Validation, verification of results, Writingoriginal and review drafts, Visualization.