Does carbon pricing reduce emissions? A review of ex-post analyses

Carbon pricing has been hailed as an essential component of any sensible climate policy. Internalize the externalities, the logic goes, and polluters will change their behavior. The theory is elegant, but has carbon pricing worked in practice? Despite a voluminous literature on the topic, there are surprisingly few works that conduct an ex-post analysis, examining how carbon pricing has actually performed. This paper provides a meta-review of ex-post quantitative evaluations of carbon pricing policies around the world since 1990. Four findings stand out. First, though carbon pricing has dominated many political discussions of climate change, only 37 studies assess the actual effects of the policy on emissions reductions, and the vast majority of these are focused on Europe. Second, the majority of studies suggest that the aggregate reductions from carbon pricing on emissions are limited—generally between 0% and 2% per year. However, there is considerable variation across sectors. Third, in general, carbon taxes perform better than emissions trading schemes (ETSs). Finally, studies of the EU-ETS, the oldest ETS, indicate limited average annual reductions—ranging from 0% to 1.5% per annum. For comparison, the IPCC states that emissions must fall by 45% below 2010 levels by 2030 in order to limit warming to 1.5 °C—the goal set by the Paris Agreement (Intergovernmental Panel on Climate Change 2018). Overall, the evidence indicates that carbon pricing has a limited impact on emissions.


Background
A recent report of the High Level Commission on Carbon Pricing and Competitiveness finds that 'Carbon pricing is an effective, flexible, and low-cost approach to reducing greenhouse gases (GHGs)' (CPLC 2017, p 8). The widespread-and growinguse of carbon pricing reflects this belief in its effectiveness. There are currently 30 carbon taxes and 31 emissions trading schemes (ETSs) across the globe, covering twenty-two percent of global emissions (World Bank 2020, p 7).
Carbon taxes place a surcharge on fuel or energy use. In ETSs, the government sets a ceiling or cap on the total amount of allowed emissions. Allowances are distributed to those firms regulated by the scheme, either free of charge or by auction. Each firm then has the right to emit up to its share of allowances. They may also trade allowances with each other to meet their individual emission allocations. Those who emit more than their allowance can purchase more; those that emit less can sell their excess supply, or bank it for future use.
Carbon taxes and ETSs differ in a number of respects. First, carbon taxes provide certainty of cost: the price is set by the government. Yet there is no limit on emissions, provided that regulated entities are willing and able to pay the tax. By contrast, ETSs provide certainty of quantity: the cap, set by the government, constitutes the upper limit on emissions. The cost will vary, depending on the scarcity (or oversupply) of allowances, and other design features. In practice, the distinction between the two policies is sometimes blurred (Hepburn 2006). For example, an ETS might have a floor price; this guaranteed price makes it resemble a tax 1 .
Second, compared with ETSs, carbon taxes are relatively easy to design and administer. Governments have lengthy experience in collecting taxes.
ETSs, on the other hand, are quite complex. Governments have to set the cap. While this is in part informed by science, it is also a function of anticipated costs. They must distribute and/or auction allowances and create a platform for tracking, trading and retiring those allowances. Often, governments auction allowances from multiple years simultaneously, which can affect future prices. If offsets are permitted as part of a carbon pricing policy, governments will need to draft or approve protocols for offset projects, which count as emissions reductions by enabling emitters to pay for decarbonizing activities elsewhere. Offsets also require a mechanism for verifying that projects actually generate the promised reductions.
Importantly, carbon pricing is not solely a domestic climate policy; it has been-and will remain-a key feature of the multilateral regime to manage climate change. The 2015 Paris Agreement creates an expanded role for carbon pricing. Article 6.2 allows countries to trade 'internationally transferred mitigation outcomes.' Essentially, a country that has exceeded the reductions outlined in its Paris pledge can sell the excess to another nation. Article 6.4 creates a Sustainable Development Mechanisma new international carbon market governed by the UN. It replaces the Clean Development Mechanism (CDM), the offset market created by the Kyoto Protocol.
And the use of international markets is not limited to the Paris Agreement. In 2016, the International Civil Aviation Organization created a new plan to address aviation emissions, which were not covered under the Kyoto Protocol. The Carbon Offsetting and Reduction Scheme for International Aviation or CORSIA scheme, will cap aviation emissions at 2020 levels by 2027. Thus, after 2027, all airlines must reduce their emissions to 2020 levels-either through offsetting or efficiency improvements. Since the scope for efficiency improvements is limited (Peeters et al 2016), the vast majority of reductions must come from purchasing offsets.
As the urgent need for action on climate change mounts, it is appropriate to ask: how well does carbon pricing perform? Do its reductions warrant the political controversies it often creates? This article looks carefully at the ex-post analyses of carbon pricing policies around the world. This is not the first review to consider the performance of carbon pricing. There are a number of other similar works, summarized below. However, this study differs from others in two key respects. First, it focuses solely on emissions reductions as the dependent variable. Unlike other reviews, it does not consider efficiency, equity, economic productivity or other criteria. Second, it conducts an exhaustive review with transparent and replicable search criteria, outlined in the following section.
The skeptic will ask: why single out carbon pricing? All climate policies face challenges. I do not dispute this fact. An in-depth comparison of policies is beyond the scope of this analysis; however, two points merit mention. First, the mismatch between the incremental effects of carbon pricing and the demand for rapid decarbonization cannot be understated. The IPCC states that emissions must fall by 45% below 2010 levels by 2030 in order to limit warming to 1.5 • C-the goal set by the Paris Agreement (Intergovernmental Panel on Climate Change 2018). The Low Carbon Economy Index estimates that this translates to an annual emissions reduction of 11.3% by the 'average' G20 nation (PwC 2019). Yet GHG emissions have risen an average of 1.5% per year in the last decade (UN Environment 2019, p iv). It is important to understand the extent to which one of the most widely-used climate policies contributes to this goal.
Second, there is little evidence to suggest that carbon pricing promotes decarbonization (Tvinnereim andMehling 2018, Rosenbloom et al 2020). Instead, the most common outcome is fuel-switching and efficiency improvements. Unlike policies which create pathways to decarbonization-such as binding renewable portfolio standards, feed in tariffs or investment in R&D-carbon pricing addresses emissions (flow), rather than overall concentrations of GHGs (stock) (Tvinnereim and Mehling 2018).
One could plausibly suggest that the relevant yardstick is how carbon pricing performs compared to other mitigation policies. Unfortunately, there are few ex-post comparisons of the reductions associated with different mitigation policies. However, extant work indicates that in jurisdictions with emissions reductions, carbon pricing is not doing the majority of the work (Egenhofer et al 2011, Wara 2014, Martin and Saikawa 2017, Cullenward and Victor 2020. Indeed, Cullenward and Victor note that 'the real work of emission control is done through regulatory instruments' (Cullenward and Victor 2020, p 10).

The politics of carbon pricing
Though increasingly widespread in their use, carbon pricing has proven to be a controversial policy, both domestically and internationally. The Paris Agreement is now 5 years old, and yet, states are still negotiating the rules for implementation (referred to as the 'Paris rulebook'). The rules on market mechanisms are the sticking point (Evans and Gabbatiss 2019).
Conflicts over carbon pricing have been even more intense at the domestic level, particularly in high-emitting developed nations. Australia, the United States and Canada, which are all global leaders in per capita emissions, have had fierce political fights over carbon pricing (Harrison 2012, Mildenberger 2020, Mildenberger and Stokes 2020).
There is a long and storied history of carbon pricing in the US, spanning from repeated failures at the federal level, to a mix of success and failure at the state level. In 1993, President Clinton proposed an energy tax (dubbed the BTU tax), which died in the Senate after considerable opposition from both Republicans and Democrats (Rabe 2018, p 46-48). Subsequent efforts to create a national cap-and-trade scheme also failed. There has been more success in creating ETSs; California and the Northeastern states in Regional Greenhouse Gas Initiative (RGGI) have had emissions trading in effect since 2012 and 2009 respectively. However, carbon taxes remain absent from US state policy (World Bank 2019). Washington state had two ballot initiatives proposing a carbon tax in 2016 and 2018; both failed following heavy investments from fossil fuel industry to defeat them.
Australia has the dubious honor of being the first developed country to repeal a carbon price. Its history of carbon pricing has been tumultuous; the policy has shifted with every change in leadership (Mildenberger 2020). And while Canada implemented a federal carbon price as part of the 2016 Pan Canadian Framework on Climate Change, it continues to tussle with provinces over the implementation of the policy, including addressing legal challenges in the Supreme Court.
Political controversies around carbon pricing are not limited to these three nations. The riots by the gilet jaunes or Yellow Vests in France were a response to an increase in fuel taxes (coupled with tax cuts for the rich), which were part of a broader strategy to reduce GHG emissions. The South African carbon tax passed after years of controversy in part because it offers generous tax-free emission allowances, ranging from 60% to 95% between 2019 and 2022 (IEA 2020).

Reviewing the literature
Reviews to date include a mixture of models and ex-post studies, and include a number of criteria in addition to, or even instead of, emissions reductions. Haites (2018) reviews the performance of carbon pricing policies based on emissions reductions and cost effectiveness (i.e. cost per ton CO2e reduced). While the paper lists 35 carbon taxes and ETSs active at the end of 2015, analysis of ex-post performance is limited to 11 nations with carbon taxes (including at the sub-national level) and 7 ETSs. He finds that overall, carbon taxes in European nations have yielded small reductions, 'up to 6.5% over several years ' (2018, p 961). But he also notes that within the EU, where nations are also part of the EU-ETS, nations without a carbon tax reduced emissions more quickly than those with a carbon tax. This finding indicates that 'other policies may have contributed more than carbon taxes to reducing non-ETS emissions.' (Ibid). The study also examines ETSs in California, the EU, Japan (Tokyo and Saitama), New Zealand and the US (the Regional Greenhouse Gas Initiative) and Switzerland. In all of these cases, Haites reports the rise or fall in emissions based on other studies, rather than whether any decrease can be causally attributed to the ETS itself. Overall, he concludes that ETSs have limited impact, since emissions have fallen faster than the cap in every jurisdiction. 2 The resulting oversupply of allowances lowers prices and undermines the effectiveness of the policy.
In a related piece, Haites et al (2018) assess the performance of carbon pricing policies along several criteria, including emissions reductions, cost effectiveness, and a number of measures of economic efficiency. They focus on whether and how tax rates have changed over time, and review the emissions reductions associated with both taxes and emissions trading systems. The study provides useful data on whether carbon pricing schemes have become more stringent over time and the extent to which various policies are associated with lower (or higher) emissions. Yet they note that they cannot disentangle the effects of carbon pricing from other climate mitigation policies (2018, p 112, 160). Narassimhan et al (2018) review eight emissions trading systems, evaluating them on the basis of environmental effectiveness, economic efficiency, market management, revenue management and stakeholder engagement. The authors create a qualitative framework to evaluate each ETS on these five criteria, including environmental effectiveness. Notably, they do not consider emissions reductions in their assessment; instead, they evaluate the proportion of emissions covered and the stringency of the emissions cap.
Other meta-reviews focus solely on one jurisdiction, and generally include a mix of models and expost evaluations. Three examine the EU-ETS (Venmans 2012, Laing et al 2014, Martin et al 2016. They draw similar conclusions; the EU-ETS produced annual reductions ranging from 0.6% to 4% for various periods between 2005 and 2012. Their estimates differ from those I present since they are not restricted to ex-post studies. Another meta-review explores various studies of the carbon tax in British Columbia, estimating that reductions between 2008 and 2014 (with some variation in dates among studies) range between 5% and 15% below a counterfactual reference level (Murray and Rivers 2015). However, they note that there are no studies that attempt to assess leakage to nearby jurisdictions. As such, they suggest that 'at least some of the reductions in emissions observed in British Columbia are likely to be associated with increases in emissions elsewhere ' (2015, p 682).
A number of books on carbon pricing also examine its effectiveness, but none provides detailed expost analyses of reductions. Cullenward and Victor examine the politics of carbon pricing, with empirical evidence drawn from policies across the globe (2020). Rabe's book Can We Price Carbon is an excellent analysis of how politics departs from economic theory where carbon pricing is concerned. He notes that policy adoption is just the beginning: 'carbon pricing policies do not necessarily self-implement and flourish' (Rabe 2018, p xvii). Without active, competent management, these efforts may fall well short of their goals. Another recent volume examines carbon markets in an impressive array of jurisdictionsranging from the EU to Tokyo to Kazakhstan (Wettestad and Gulbrandsen 2017). The volume provides useful insights on the history, and design of carbon pricing, but relatively little on its functioning. In sum, there is much more work to be done to evaluate the actual performance of carbon pricing policies.

Methods
To compile the list of studies, I used a systematic review process, supplemented with a snowball approach, ensuring the broadest search possible. I began with citations in Google scholar, 3 using the following search terms: • carbon tax emission effects • emission trading emission effects • effects ETS • carbon pricing effectiveness • carbon pricing leakage • 'cap-and-trade' emission effects • carbon tax emission Searching from 2000 to the present, I then reviewed all articles in the first ten pages returned in each keyword search. I included only those articles that meet several criteria. First, the paper must provide a quantitative evaluation of emissions reductions in a given jurisdiction. Moreover, papers must employ some type of causal inference, which seeks to isolate the amount of emissions reductions attributable to the carbon pricing policy. Inference is conducted most frequently through regression models, matching techniques and synthetic controls. Regression models generally control for a variety of factors, such as energy prices, the presence of renewable portfolio standards, feed in tariffs, fossil fuel subsidies, among others. Matching studies compare emissions in regulated and unregulated jurisdictions which are comparable in other attributes. Synthetic control studies compare observed emissions to a hypothetical comparable jurisdiction, generally created by a weighted combination of similar jurisdictions without the carbon pricing policy. Papers which simply demonstrate co-variation between pricing policies and emissions levels are not included, since they do not analyze whether observed changes in emissions result from the policy enacted.
Second, the paper must be an ex-post evaluation of the performance of the carbon pricing policy. I therefore exclude simulations, predictive models or theoretical assessments of reductions. It should be noted that these prospective analyses constitute the vast majority of the quantitative literature on carbon pricing.
Third, the dependent variable for the study must be emissions reductions. In most cases, studies estimate emissions reductions in the sectors covered by the carbon pricing policy, though some extrapolate to broader jurisdictional effects (e.g. Murray and Maniloff 2015, Bayer and Aklin 2020. Papers that examine reductions in consumption of fuel or electricity are not included unless the reductions are quantified in terms of CO2e emissions. Similarly, works that estimate changes in investment decisions or innovation outcomes are excluded, since these are only indirect measures of emissions reductions. This coding criterion also allows for studies that evaluate leakage, provided that leakage is quantified into emissions (rather than say, flows of goods or electricity). The decision to limit the scope of the dependent variable was made to facilitate comparison across studies. In general, the studies share a similar model. Emissions (defined various ways) are the dependent variable; the carbon pricing instrument is the independent variable.
Fourth, articles are included if they are peerreviewed. Grey literature is also included according to the same criteria. Eight of the papers (22%) are grey literature, which includes papers published by the World Bank and the OECD, as well as think tanks such as the Institute for Climate Economics, and working papers published by universities. I have also included one scholarly working paper which meets the other criteria but has yet to be published (Pretis 2020).
I exclude governments' evaluations of their own programs as part of the systematic review for two reasons. First, while some such reviews are undertaken by independent, third-party agencies, this is not uniformly the case. Thus, it is difficult to gauge which evaluations might be more or less neutral without in-depth knowledge of the governmental structures of each jurisdiction. Second, it is difficult to conduct a systematic review of these evaluations, since there is no central database or source to query. As a result, it is more likely to unintentionally exclude some evaluations, which could skew the results.
Using these criteria, articles were initially coded by a research assistant, with final adjudication by the author. This initial process yielded a total of 12 articles. There were also six meta-reviews which assessed the performance of one or more carbon pricing policy based on others' studies.
I then switched to a snowball sample, reading all 18 articles (the 12 qualifying articles and the six metareviews) for additional citations. This yielded another 30 articles. Thus, I reviewed a total of 48 articles to see if they would be included in this analysis. Of the 48 reviewed, a total of 37 met the criteria outlined above, and were included in the final analysis. Eight were meta-analyses which were read for citations, but not coded in the section 5, since they did not conduct independent ex-post analyses.
These are strict criteria, which yielded a relatively modest number of articles. However, this narrow approach is important, since ex-post evaluations are the only analyses that can really contribute empirical evidence to inform policymaking. There are thousands of scholarly articles on carbon-pricing, but the vast majority operate in the realm of what if, rather than what is. This finding is consistent with Barry Rabe's book Can We Price Carbon, which shows that theory and practice are quite far apart across several cases of carbon pricing, separated by the pesky problem of politics (Rabe 2018).

Results
The 37 studies compiled in this review reveal five key findings. First, it is astonishing how little hard evidence there is on the actual performance of carbon pricing policies using ex-post data. This point cannot be understated. It is the collective consensus that we need carbon pricing to address climate change, but the reality is we have very little evidence to substantiate this claim. Even carbon pricing policies with broad coverage, such as Japan and California, lack extensive independent evaluations. Second, the overall effect on reductions for both types of policy is quite small, generally between 0% and 2% per annum. Third, on the whole, taxes appear to do slightly better than ETSs in producing reductions. Fourth, the impact of the EU-ETS-the largest and oldest international carbon market-has been extremely limited. Finally, the highest emissions reductions estimates are from studies using the synthetic control method. I address each point in turn. Figure 1 provides an overview of the studies. Panel (a) shows that geographical span of these studies skews heavily, and unsurprisingly, toward Europe. The Euro-centric focus is partially a function of history: Europe has the longest record of carbon pricing. Norway, Sweden and Denmark were early adopters, implementing some of the first carbon taxes in 1991-92 (World Bank Group 2020). And the EU-ETS was the first compulsory ETS, beginning in 2005. Panel (b) demonstrates that the majority of studies focus on ETSs. Figure 1 also highlights important omissions. The Japanese carbon tax, enacted in 2012, covers roughly 65% of the nation's emissions, and yet, has no posthoc evaluations-though it is possible that they are only available in Japanese. One would imagine, however, that such a tax covering such a broad swath of the economy would be of interest to scholars of carbon pricing globally.
Similarly, there are surprisingly few studies of the California ETS (see table 1). The California program began in 2013, and now covers nearly 75% of the state's emissions 2020 (The Legislative Analyst's Office, an independent assessment agency of the California government, has conducted a number of evaluations of the state's climate policies. In a report on electricity generation, it notes the absence of scholarly studies of the effects of cap-and-trade on emissions and concludes: 'Based on conversations with stakeholders and researchers, the effect on electricity sector emissions is generally thought to have been relatively modest compared to other policies, such as RPS'  Panel data w/comparison to firms outside Tokyo w/fixed effects  Panel data w comparison to other nontaxed provinces, using fixed effects Shmelev and Speck (2018) 1960-2010 Sweden General carbon tax had no effect on aggregate emissions, except in the case of petrol, but separate taxes on coal and petroleum gas did reduce emissions.
Time series (Petek 2020, p 19). It further notes that these effects are likely reduced by the effects of leakage (Ibid). Beyond the incorporation of Iceland, Lichtenstein and Norway, the EU-ETS, there is little research on the effects of linking carbon markets. California and Quebec market linked their markets in 2014, yet I was unable to find any scholarly assessments of their joint performance that meet the criteria outlined above. A 2016 evaluation by an environmental NGO IQCarbon does not isolate the causal effects of the linked programs, but notes that price volatility is problematic, and linked to political uncertainties in California. These problems have since been addressed by the renewed, more ambitious commitments (Didioti and Purdon 2016). The lack of post hoc analysis on linked markets is particularly surprising, given that a number of scholars have called for linking carbon markets across jurisdictions as a way to coordinate climate policies, lower costs under certain conditions (Doda and Taschini 2017) and reduce opportunities for leakage ; however others have cautioned against this approach (Green 2017).
The second key finding is that overall, emissions reductions from both types of carbon pricing policy are limited-in the low single digits per year. The EU-ETS, the largest and oldest ETS, is a most likely case for the success of carbon pricing. Yet, overall emissions reductions across all sectors in the EU-ETS range between 0% and 1.5% per year; it is important to note that some of these estimates include the first phase of the EU-ETS, which was considered to be a pilot phase. The single study of California cap and trade scheme estimates that between 24% and 43% of emissions from electricity generation were shifted out of state to avoid carbon pricing regulations (author's calculations based on Cullenward 2014) 4 . The RGGI, an ETS in the Northeastern United States, appears to be quite effective-reducing electricity emissions by 19% over 4 years (Murray and Maniloff 2015). However, it is difficult to parse the effects of the ETS from other parts of the program such as energy efficiency measures and low carbon investments (Ibid, 588). Moreover, there are concerns about leakage, which are discussed further below.
Third, carbon taxes tend to produce more reductions than ETSs (see table 2). The carbon tax in British Columbia has reduced emissions somewhere between 5% and 15% between 2008 and 2015 (Murray and Rivers 2015)-or a little less than 2% per year, using the most optimistic estimate (see also Schaufele 2015, Pretis 2020). The UK carbon pricing policies also stand out as having achieved larger reductions compared to other policies in this review. One study finds that the UK carbon price support reduced emissions in the power sector between 41% and 49% over 4 years (2013-17) (Leroutier 2019). Another finds it reduced overall emissions by 6.2% between 2013 and 2016. The success is likely due to the drastic reduction in the use of coal-fired electricity (Cullenward and Victor 2020, p 3). The earlier UK Climate Change Levy reduced emissions of plants paying the full rate between 8.4% and 22.6% compared to plants that paid the reduced rate .
Nordic taxes also tend to do better on reductions, though the wide variation in findings makes it difficult to conclude this definitively. Sweden was one of the first nations to introduce a carbon tax in 1991, and the current price of US$119 per ton is the highest in the world (World Bank Group 2020). In a recent study, using a 'synthetic Sweden' as a basis for evaluating the effect of the carbon tax, Andersson finds that the tax reduced emissions by 6.3% in an average year (Andersson 2019, p 3). He notes that his finding is a departure from previous studies, including three in this review, which find that the Swedish tax has little to no effect on emissions. Bohlin finds small reductions in district heating emissions, but none in other sectors (1998). Shmelev and Speck conclude that petrol emissions were the only reductions achieved by the carbon tax (2018). Lin and Li study taxes in Denmark, Finland, Netherlands, Norway, Sweden. Using a difference-in-difference approach, they find only the Finnish tax reduced the per capita growth rate of emissions by 1.7% (2011).
It is important to note that one outlier among the Swedish studies is in the grey literature. Using the synthetic control method to estimate the effects of carbon taxes in Nordic countries, Fernando estimates that the carbon tax in Sweden has caused an average annual reduction of 17.2%. She finds a similarly large reduction in Norway (19.4%), though no statistically significant effects in Denmark or Finland (2019).
Why might taxes do a better job at emissions reductions compared to ETSs? Data from I4CE show that 79% of carbon taxes are imposed at the national level. By contrast, only 44% of ETS occur at the national (or in the case of the EU, supranational) level (I4CE 2020). Thus, many ETS that occur without the support of the federal government, potentially diminishing what they may be able to accomplish. Second, political scientists suggest that firms tend to back carbon trading over taxation, since they view it as a less costly form of regulation (Meckling 2011). Depending on program design, the possibility for free allowances and offsets can further reduce potential impacts on business as usual. This logic is further supported by recent research which finds that 'carbon intensive economies tend to prefer emissions trading over carbon taxes' (Skovgaard et al 2019, p 1173). Thus, a political explanation for the relative performance of each carbon pricing instrument lies in the relative influence of industry in policy design and adoption.
The third key finding is that the EU-ETS, has had a very limited impact. The EU-ETS is arguably a mostlikely case for success. It is administered by wealthy nations with extremely high regulatory capacity. It has undergone three phases, which have allowed for learning and adjustment over time. And the market is now carefully regulated by the European Commission through the Market Stability Reserve, which adjusts the supply of allowances to avoid oversupply and absorb exogenous shocks to the market.
Despite the extensive human and financial resources invested in developing and managing the EU-ETS, annual emissions reductions (i.e. across all sectors) range between 0% and 1.5% per year. Four studies found no discernible effect of carbon prices in Phase 1 (Gloaguen and Alberola 2013, Petrick and Wagner 2014, Jaraite-Kažukauske and Di Maria 2016. To be fair, this is unsurprising given that Phase 1 was the pilot phase of the ETS, and essentially allowed states to set their own caps. Indeed, Anderson and DiMaria find that in Phase 1, total allocation of allowances was only 0.45% below business as usual (2011). Thus, their finding of 2.8% net emissions abatement should be considered taking into account the generous caps. Since Phase 1 was primarily meant as a learning phase, its failure to reduce emissions should not be construed as a policy failure. Moreover, the inclusion of Phase 1 in longer studies skews overall effects downward. Some studies include some or all of Phase 2, and the effects vary widely-largely depending on the sectors included. For example, Bayer and Äklin find that the EU-ETS reduced emissions by 3.8% of EU's total emissions between 2008 and 16 (Bayer and Aklin 2020). While their estimate cannot be readily averaged, it translates roughly to 0.5% average annual reduction. Similarly, Dezchelprestre et al estimate that the EU-ETS reduced emissions of regulated installations by 10% between 2005 and 2012, compared to non-regulated ones (2018). In sum, for those studies that calculate average effects across the EU, reductions range from 0% (Gloaguen and Alberola 2013) to 3.1% over 2 years (Ellerman and Buchner 2008). Again, it is important to note that with the exception of Bayer and Äklin, most studies include Phase 1, which will skew findings of total reductions downward.
Certain sectors appear have more substantial reductions under Phase 2 of the EU-ETS. Petrick and Wagner (2014) find that German manufacturing firms reduced their emissions between 25% and 28% relative to unregulated firms between 2008 and 10. French manufacturing firms reduced emissions between 13.5% and 19.8% in the same time period, largely due to fuel switching .
Studies of emissions intensity find marginal improvements, suggesting that the ETS promotes some degree of fuel switching. Egenhofer et al find an average annual intensity improvement of 3.35% in regulated sectors in Phase II, compared to 1% in Phase 1 (2011). Ellerman et al (2016) estimate an average decline of 3% in emissions intensity between 2004 and 2014, compared to a 1% reduction before the ETS took effect.
Given that studies vary in their time periods, countries and sectors, it is not possible to ascertain the overall reductions produced by the EU-ETS. This is further exacerbated by the fact that isolating the causal effects of the ETS is difficult, as many authors note. However, three trends are clear. First, the overall reductions are quite low, ranging from 0% to 1.5% per year.
Second, the largest reductions are limited to a specific sector or sectors; they do not refer to economywide reductions. For example, Dechezlepretre and colleagues (2018) estimate that the EU-ETS resulted in a 10% reduction in emissions between 2005 and 2012 (though they caution about the generalizability beyond the four countries studied). This should be interpreted as a 10% reduction in among the regulated sectors-which comprise about 45% of emissions within the EU (World Bank n.d.). Similarly, Wagner et al (2014) find that French manufacturing firms reduced emissions between 13.5% and 19.8% between 2008 and 2012, but again, this only applies to the regulated sector. This is consistent with recent work by Cullenward and Victor, which emphasizes the advantages of a sectoral approach to decarbonization (2020).
Third, the drivers of these modest reductions are incremental solutions: fuel switching, enhanced efficiency, and reduced consumption of fuels (Tvinnereim and Mehling 2018). These actions, though useful on the margins, fall well short of the societal transformations identified by decarbonization scholars (Unruh 2000, Bernstein andHoffmann 2018).
The final point is methodological. There are a diversity of methods used in the studies.
Interestingly, some of the highest estimates of emissions reductions across jurisdictions are studies that use synthetic control methodology. As noted above, Anderson estimates an average annual reduction of 6.3% in Sweden. Fernando's estimate is almost three times that. In her unpublished study of the UK Carbon Price floor, Leroutier uses synthetic controls and finds that the policy reduced power sector emissions between 41% and 49% between 2013 and 17. However, not all synthetic control studies produce such high estimates. For example, Bayer and Aklin (2020) find a modest 3.8% reduction in emissions in the EU-ETS over 8 years.

Discussion
In this section, I provide a broader context for understanding the effectiveness of carbon pricing. First, I discuss whether the limited reductions are simply the product of low prices. Second, I address additional reasons why these reductions might be overestimated: the twin problems of leakage and offsets. Third, I consider the political responses to carbon pricing policies.
A common rejoinder is that carbon prices simply are not high enough to generate substantial emissions reductions. Indeed, low prices are pervasive; the vast majority of carbon prices are well below even the most conservative estimates of the 'social cost of carbon' (SCC). The SCC internalizes the environmental and health effects of GHG emissions. A recent study surveyed environmental experts on their estimation of SCC, which ranged between $80 and $300 ton −1 (Pindyck 2019). Another study estimates a global median price of $417, with substantial national level variation (Ricke et al 2018) A more conservative estimate puts the SCC between $50 and $100 by 2030 (Carbon Pricing Leadership Coalition 2017).
Even compared to the most conservative estimates of the SCC, carbon pricing falls short. The most recent World Bank survey of carbon pricing shows that half of the 61 carbon pricing policies around the globe have a price lower than $10. The IMF estimates that the average global price for carbon is $2 ton (Parry 2019).
Given the prevalence of low prices, it is particularly important to consider the few jurisdictions with carbon prices at or near the SCC. As noted above, Sweden has the highest carbon price in the world. Studies range in their reduction estimates from 0% to 17% per year, with the upward bound being an outlier among all 37 studies. In 2019, Finnish taxes on transport fuels were at $68 ton, and $58 ton for all other fossil fuels. Emissions reductions there are estimated to be between 0% and 1.7% (Fernando 2019, Lin andLi 2011). The other two jurisdictions with high carbon taxes are Switzerland ($99 per ton in 2019) and Lichtenstein ($99 per ton in 2019); I was unable to find any estimates of their effects on emissions.
It may be the case that pricing will work better after a certain threshold is surpassed. Indeed, Aydin and Esen find that energy taxes, including CO 2 taxes, only reduce emissions after surpassing 2.2% of GDP (2018). Yet after nearly four decades of experience with carbon pricing, the empirical evidence to date suggests that low prices are a feature of this policy, rather than a bug. More worrisome is the fact that even those nations with high prices have relatively modest reductions.
A second potential problem for carbon pricing concerns leakage, which occurs when economic activity subject to carbon pricing shifts to a jurisdiction without similar regulations. This problem is pervasive in environmental regulation, driven by variation in policy stringency (see, e.g. Vogel and Kagan 2004). This is particularly true when capital is highly mobile. Carbon pricing is no exception. Thus, leakage may result in a relocation of emissions, rather than a net reduction.
Although about half of the studies (46%) mention leakage, they do not incorporate it explicitly into their models. There is an obvious methodological explanation for this: estimating leakage is extremely difficult. It requires estimating BAU emissions for a given sector or facility, and then identifying specific transactions (often energy generation) that have changed after the implementation of carbon pricing. Add these calculations to those made to estimate emissions reductions due to carbon pricing, and the overall analysis becomes extremely complex. To the extent that leakage occurs, but is excluded from the studies examined here, emissions reductions may be overestimated.
A handful of studies in this review explicitly tackle the problem of leakage in California, the EU and in RGGI. California appears to have a major problem with leakage. In evaluating individual contracts for four power plants, Cullenward (2014) estimates that between 2009 and 2012, two plants leaked between 22.0 and 39.0 Mt CO 2 e (see also Caron et al 2015). For reference, average annual emissions from electricity across those years was roughly 90 MtCO 2 e (California Air Resources Board 2019, p 9).
Like the California cap-and-trade scheme, RGGI has neighboring states that are not part of the ETS. As a result, shifting electricity generation outside of the regulated jurisdictions becomes easier, and leakage may result. One study finds that RGGI has produced considerable reductions: electricity emissions in RGGI states are 19% lower than they would have been in the absence of the ETS (Murray and Maniloff 2015). However, the authors do not consider the possibility of leakage in this analysis. A subsequent study finds that though RGGI produced annual reductions of 8.8 Mt CO 2 , surrounding states increased emissions by 4.5 Mt CO 2 annually (Fell and Maniloff 2018). This data suggests that leakage seriously undercuts the effectiveness of this program.
A few studies of the EU-ETS consider the problem of leakage, though they do not provide any estimates. One study reaffirms the approach taken in the EU-ETS to distribute free allowances to those firms facing stark competition due to carbon pricing.
They note that firms in trade-exposed sectors performed better than equivalent non-ETS firms, which they interpret as evidence of the effectiveness of free allowances (Dechezlêpretre et al 2018, p 13). Another reason that the decline in emissions in energy reductions is not likely to be the result of leakage, since energy production is 'fairly immobile due to a large share of fixed assets' (Bayer and Aklin 2020, p 6). A third finds no evidence of within-firm leakage, and therefore posits, by assumption, that leakage outside the EU-ETS market is unlikely. However, it does not provide direct evidence for this claim . In sum, there is limited consideration of the issue of leakage in the EU, which suggests that it is unlikely to be large problem-at least for the most exposed sectors. This is consistent with the geographic breadth of the policy, which reduces opportunities for leakage. Unlike California, where neighboring states (that share an electricity grid) are not regulated by a carbon price, the span of the European market makes this strategy more difficult.
To fully understand the effects of carbon pricing, one must also consider the role and robustness of offsets. Offsets allow regulated entities to meet some or all of their compliance obligations by paying for emissions reductions elsewhere. The reductions are quantified against a hypothetical counterfactual: the emissions that would have occurred in the absence of funding for the project. The additional reductions are referred to as a project's additionality. Measuring additionality is a difficult endeavor for a number of reasons beyond the hypothetical counterfactual (Gillenwater et al 2007).
Offsets can have two possible impacts on overall reductions. First, to the extent that offsets are not additional, their use will decrease the actual reductions achieved through a carbon pricing policy. Such an assessment would require knowing the extent to which a given project or offset methodology is not additional, and the number of credits claimed for that project or protocol under a specific carbon pricing policy. Second, those regulated entities that rely more heavily on offsets will have fewer in-situ reductionsthus contributing to the relatively small reductions documented in this analysis. In both instances, the overall effect on emissions relies heavily on offset quality. As the discussion below illustrates, there are legitimate reasons to be concerned about the quality of offsets and the extent to which they represent additional reductions.
To date, offsets have been an important component of most ETSs. The EU allowed up to 50% of EUwide reductions to come from offsets in Phases 2 and 3, largely from the CDM of the Kyoto Protocol (ICAP 2020a). Yet the CDM was rife with problems. One study estimates that 73% of emissions reductions generated by the CDM between 2013 and 2020 'have a low likelihood that emission reductions are additional and are not over-estimated' (Cames et al 2016, p 11). The heavy reliance on CDM credits in the EU-ETS surely affected the total reductions under the EU-ETS. Due in part to this problem, as well as a number of others (Wara 2007), the EU has limited the size and scope of eligible offsets. In 2013, it disallowed industrial projects to destroy HFC-23, and required projects to take place in the developing world. As of 2020, it has discontinued the use of international offsets generated by the CDM.
The ETS linking Quebec and California permits up to 8% of allowances to be generated through offsets. California offsets are limited to the US, but a series of policy and scholarly papers raises questions about their additionality. One study estimates that 82% of the credits generated through improved forestry management do not represent genuine reductions (Haya 2019). Another suggests that Californian offset protocols have reduced, but not eliminated, problems of over-crediting.
More generally, we should recognize that offset reductions are often problematic. Because offsets require calculations against a hypothetical counterfactual, they are always subject to measurement problems. As a result, a recent analysis argues that in California, 'it may be more useful to think of offsets as government-intermediated incentive programmes in which regulated emitters are allowed to invest in lieu of reducing their own emissions' (Haya et al 2020, p 15). In Québec, all projects generating offset allowances are located in the province. There do not appear to be any studies evaluating the performance or additionality of the handful of offset projects in the province 5 .
The RGGI Model Rule allows each plant to use offsets to fulfill up to 3.3% of its compliance obligations, though this is not uniform across all states (RGGI 2021). Five different types of projects are eligible, though two-sulfur hexafluoride and enduse energy efficiency-will become ineligible beginning in 2021. According to the International Carbon Action Partnership, a forum for exchange among governments and other actors participating in emissions trading, there is only one carbon offset project currently active under RGGI (ICAP 2020b).
Finally, though not an ETS, CORSIA, the aviation emissions reduction agreement, will rely heavily on offsets to achieve its goals. Estimates for the demand for offset credits range from 1.6 to 2.5 billion tonnes CO2e (EDF and IETA 2016, Carbon Watch 2020) between 2021 and 2035. A number of studies affirm that there are ample credits available. Importantly, this is due in part to the fact that ICAO, which governs the CORSIA agreement, has recently decided to accept offset credits from the CDM (2016 vintage and forward) (ICAO 2020). It will also accept a number of offsets from the voluntary market, including the American Carbon Registry, the Climate Action Reserve, the Gold Standard and the Verified Carbon Standard (ICAOI 2020). Thus, in the absence of a credible decarbonization strategy, the aviation sector is 'all in' on offsets-a carbon pricing instrument with numerous documented problems.
These discussion points illustrate that offsets encounter numerous challenges, and these will most likely negatively affect the estimated reductions of any ETS. As Cullenward and Victor note, there is simply no constituency for high quality offset projects (2020). Virtually everyone involved-from the regulated entity seeking to achieve compliance to the project verifier-has an incentive to move projects forward (Green 2014, ch 4). Quantity takes precedence over quality. And the incentive to find low-cost projects increases the likelihood of non-additionality (Ibid). Opponents of offset projects, often environmental NGOs and environmental justice organizations, are generally outside the project process. In sum, while it is not realistic to expect that an ex-post evaluation of carbon pricing will also consider the difficult problem of evaluating offset additionality, it is critical to recognize their effects on estimates of overall reductions.
Finally, while this study has focused on emissions reductions, the political challenges of carbon pricing cannot be overlooked. It is clear that carbon pricing is a controversial policy in many highemitting developed nations (Jenkins 2014, Rabe 2018, Mildenberger 2020). There are two sources of this opposition. First, high emitting industries are well-organized and powerful, and are able to use their extensive resources to block progress on climate policy, including carbon pricing (Mildenberger 2020, Stokes 2020, Colgan et al 2021) Second, public opinion research indicates that publics tend to prefer other policies over carbon pricing. Some have suggested that revenue neutral taxes can address this opposition, since they redistribute the revenue back to taxpayers. However, some work indicates that revenue-neutral taxes do not always alleviate these objections (Dolšak et al 2020, Mildenberger et al 2021. Similarly, tax-and-dividend policies appear to be the best way to address opposition (Carattini et al 2019). In this approach, revenues raised from carbon taxes are recycled to the public, and ideally, in a progressive manner, so that lower-income households receive greater dividends. Yet it is far from clear that such redistribution would assuage objections to more taxation. Indeed, most studies find that the public is more supportive of green investments than a tax-and-dividend policy (see e.g. , Bergquist et al 2020, Douenne and Fabre 2020. Many politicians have also painted carbon pricing in a negative light. The Premier of Ontario not only cancelled the cap-and-trade scheme upon his election, he also required gas stations to post stickers about the cost of the federal carbon price on gas pumps (this was recently found to be unconstitutional). In short, it is not at all evident that limited political capital should be spent on carbon pricing when other efforts at mitigation may offer more reductions for less political controversy.

Conclusion
For a policy that has dominated much of the discourse in climate politics, the analysis here demonstrates that collectively, we know relatively little about its ex-post performance, and what we do know is concentrated in a few jurisdictions. The available information indicates that its impact on emissions has been limited at best. This is worrisome for both political and efficacy reasons. First, in terms of efficacy, there is a strong argument to be made that emissions reductions should be much more heavily weighted against other evaluative criteria. The IPCC has indicated the urgent need for more ambitious reduction goals. And the pledges under the Paris Agreement are nowhere near sufficient to limit warming to 2 • C (UN Environment Programme 2019). And there are reasons to believe that the rate of climate change will continue to accelerate (Xu et al 2018). At best, carbon pricing has produced incremental reductions. If it is to be used as a tool for mitigation, it should only be used in tandem with other, more aggressive policies.
Third, there are large international regulatory implications for the performance of these domestic policies. Both Article 6 of the Paris Agreement and the recent ICAO agreement on aviation emissions indicate create a demand for an expanded international carbon market including linking domestic carbon markets and trading credits for international offsets. The Negotiations about the implementation of Article 6 have been contentious; despite inking the Paris Agreement 5 years ago, rules on market mechanisms remain unresolved. Policymakers should think carefully about further developing global markets given the limited impacts of carbon pricing. Similarly, they should approach linking different markets with caution. Linkage is a complex regulatory endeavor, which may introduce unintended consequences and make problems harder to correct (Green 2017). Such an approach might be warranted if it were to produce large reductions in emissions, but thus far, there is little evidence to support this claim.
Future research in three areas would be particularly helpful in informing policy discussions.
First, much more ex-post empirical work on the effect of carbon pricing on emissions reductions is needed-particularly in nations which have lower regulatory capacity. Isolating the causal effects of carbon pricing versus other climate policies is difficult (Egenhofer et al 2011). More studies will help validate the accuracy of current estimates. Moreover, of the small corpus of studies on carbon price performance, the vast majority are in the developed worlda most likely case for success given the higher levels of regulatory capacity. It is possible that subsequent policies will learn from previous ones, but only further research can confirm or reject this hypothesis. Second, further research should investigate whether and how carbon pricing contributes to political progress or polarization on decarbonization. Some suggest that carbon pricing should be used in tandem with other policies. But public opinion tends to support carbon pricing less than investment in renewable energy and other climate policies (Bergquist et al 2020). Additional research can help policymakers understand whether it is politically feasible to include carbon pricing as part of an 'all of the above' approach. Third, comparative statics would help. Though measurement would be challenging, it would be useful to know how carbon pricing stacks up against other mitigation approaches in ex-post analysis of emissions reductions. More data on the relative contributions of different policies to short-term emissions reductions could help prioritize the use of political and financial resources.

Data availability statement
All data that support the findings of this study are included within the article (and any supplementary files).