Making incremental progress: impacts of a REDD+ pilot initiative in Nepal

Reducing emissions from deforestation and forest degradation (REDD+) encompasses a range of incentives for developing countries to slow, halt and reverse forest loss and associated forest carbon emissions. Where there is high dependence on biomass energy, cleaner cooking transitions are key to REDD+’s success. Given the poor track record of efforts to promote clean cooking, more evidence is needed on the potential for REDD+ to reduce unsustainable extraction of biomass energy. We present a quasi-experimental impact evaluation of REDD+ in Nepal. Unsurprisingly, we find little evidence of impacts on forest carbon in just two years. We do find that REDD+ reduced forest disturbance as measured by four plot-level indicators (signs of forest fire, soil erosion, encroachment and wildlife) that are predictive of future changes in net carbon emissions and reflective of reduced extraction pressure by households. While our analysis of household survey data does not show that REDD+ reduced harvest of forest products, we find some evidence that it reduced household dependence on firewood for cooking, possibly by increasing use of biogas. Thus, communities in Nepal appear to have improved conditions in their forests without undermining local benefits of those forests. To secure progress towards reduced emissions and improved livelihoods, interventions must be designed to effectively meet household energy needs.


Introduction
Reducing Emissions from Deforestation and Forest Degradation (REDD+) has emerged as a leading near-term option to mitigate global climate change.
In addition to reducing carbon emissions, REDD+ is expected to 'incentivize protection and conservation of natural forests and their ecosystem services, and to enhance other social and environmental benefits' (FCCC/CP/2010/7/Add.1). While REDD+ has yet to achieve its potential as the world's largest system of payments for ecosystem services (Corbera 2012), there has been an unprecedented flow of global resources directed at REDD+ readiness and pilots (Sills et al 2014). These pilots do not necessarily offer direct conditional payments (Wunder et al 2020), but a subset do make conditional payments to communities who choose how to reduce emissions and how to allocate payments, similar to community demand driven programs in other sectors (Pattanayak et al 2010, Mansuri andRao 2004). Evaluation of these pilots thus needs to consider the causal mechanisms, or intermediate outcomes, selected by the communities themselves. Rigorous evaluation of these intermediate outcomes, as compared to counterfactual scenarios, can help meet immediate needs for information about the potential of REDD+ to mitigate climate change and the risks that REDD+ will undermine livelihoods (Caplow et al 2011, Simon et  We present findings from an evaluation of the intermediate outcomes of a short-lived pilot REDD+ initiative in Nepal that made payments in 2011-2013 to community forest user groups (CFUGs) in three watersheds (figure 1). There are good reasons to pilot REDD+ in Nepal, such as the country's high dependence on biomass fuels and experience with community-based forest management. In the pilot that we evaluated, the CFUGs sought to reduce net forest carbon emissions by reducing forest degradation. To meet this objective, they focused on two causal mechanisms: household energy transitions and improved forest management. Specifically, within the CFUGs, payments were disbursed by management committees in the form of low (even zero) interest loans to households for adoption of biogas and improved cookstoves (Shrestha et al 2014). In addition, the REDD+ pilot raised awareness, promoted improved forest management and offered income generation opportunities. Because of space constraints, we provide additional background and context in the supplementary information (SI) section S1 (available online at https://stacks.iop.org/ ERL/15/105004/mmedia).
The intermediate outcomes on the causal path to the ultimate objective of reducing net forest carbon emissions as well as generating ecological and social co-benefits are portrayed in figure 2. They include increasing adoption of improved cooking technologies and reducing harvesting pressure on forests. Thus, this community demand driven REDD+ pilot incorporated a long-standing strategy for reducing carbon emissions by reducing unsustainable harvest of biomass for household cooking, which has been used to generate offsets for the Clean Development Mechanism and the voluntary market (Simon et al 2012, Freeman and Zerriffi 2014. However, there has been relatively little attention given to the connections between this offset strategy and REDD+, even though they both rely on reducing forest loss as a means to mitigate climate change (Hofstad et al 2009). Understanding the potential synergies and trade-offs between reduced emissions and local cobenefits presented by this strategy can help inform national plans under the Paris Agreement and align them with the 2030 Agenda (Vivid Economics 2019).
We evaluate the impacts of the REDD+ pilot using a quasi-experimental design, collecting data from households and forest plots in CFUGs in the three watersheds selected for the pilot and comparison CFUGs matched on observable characteristics (Jagger et al 2009, Sills et al 2017. We collected data immediately before (2011) and in the last year of the intervention (2013) on forest carbon, ecological health, and household cooking technologies and forest use. We analyzed differencein-differences (DID) to control for time-invariant ('fixed') unobserved differences between the REDD+ (or treatment) and comparison communities.
We detect little effect on forest biomass (statistically significant only for litter) and no impact on total forest carbon. However, we find that REDD+ reduced forest fires, soil erosion, and encroachment, which are all key 'predictive proxy indicators' (Miller et al 2017) of forest degradation and carbon loss. We thus demonstrate incremental progress towards the ultimate goal of reducing net forest carbon emissions. REDD+ also increased wildlife sightings, suggesting co-benefits for biodiversity. Turning to possible intermediate outcomes of REDD+ that could result in reduced forest degradation, REDD+ decreased the share of cooking done with firewood, even though it did not affect the number of firewood loads collected perhaps because households increased their energy use when they installed biogas systems. In sum, it appears that in the short-term, these communities were able to use the REDD+ intervention to improve forest conditions without undermining local benefits from forests.

Methods
Our quasi-experimental impact evaluation combines statistical matching with DID estimation Jones and Lewis (2015). Our study design is the 'Before After Control Intervention (BACI)' research design that has been used for assessing environment and development policies (Pattanayak et al 2010) and recommended for REDD+ interventions (Sills et al 2017). This design allows us to account for observable and time-invariant unobservable factors and thus attribute changes in outcomes to the REDD+ intervention (Pattanayak 2009, Sills et al 2017. We were able to collect baseline, or 'before,' data, because we initiated our evaluation in conjunction with the design stage of the REDD+ pilot project.

Quasi-experimental design and sample selection
Because the REDD+ treatment was applied to CFUGs, i.e. CFUGs received payments based on their characteristics and performance, we designed our study to control for any systematic differences in the CFUGs that were included in the pilot. We estimate impacts on households and plots within those CFUGs, similar to an 'intention to treat' estimator in an experiment that selects units for treatment but does not achieve perfect compliance (Gupta 2011). The first step in our research design was to identify a sample of statistically matched treatment and control CFUGs, i.e. CFUGs included in the REDD+ pilot and other CFUGs in the same district with similar characteristics. The pilot included all CFUGs in three watersheds in Chitwan, Gorkha and Dolkha districts. We selected matched samples of 'treated' CFUGs in these watersheds and 'control' CFUGs in the same districts through three steps (details in supplementary information, SI section S2):  (a) identify initial pool of 84 treated and potential control CFUGs with broadly similar biophysical and socioeconomic characteristics through key informant consultations in the three districts; (b) collect data on 11 community-level indicators in each of these CFUGs through rapid rural appraisal and review of administrative records (table S1); and (c) select final set of 42 CFUGs (21 treated and 21 control) through statistical matching, using the 11 indicators to undertake covariate (Mahalanobis distance) matching.
Our final sample includes seven treated CFUGs in each watershed, which we matched with seven control CFUGs in different watersheds but the same districts, for a total sample size of 42 CFUGs (figure 1). The community-level indicators used for matching were balanced across the resulting sample of 21 treated and 21 control CFUGs. We also updated five of those community-level indicators using the baseline household survey data. Table S2 shows that balance was achieved on most of the updated indicators (no statistically significant differences in treated and control CFUGs). However, the survey data show that there were significantly more local organizations (NGOs) and a significantly higher fraction of households using liquid petroleum gas (LPG) to cook in the treated than in the control CFUGs at baseline. We therefore include these variables as covariates in the post-matching regression models (see equation (1) below). In addition, the baseline data reveal significant differences in most of the outcome variables (table S3), suggesting other unobserved differences between treatment and control CFUGs. This confirms the importance of the BACI study design, which allows us to control for timeinvariant differences using the DID estimator.

Data
We collected data on forest biomass and ecological indicators (table S3) in a total of 554 plots, including 124 plots in control CFUGs (during July and August, 2011 and 2013) and 153 plots in treated CFUGs (during February to May, 2011 and 2013). These plots were on average slightly more than 30 min walk from the survey households (described below). This is similar to the time that those survey households reported walking to forests where they collect firewood, grass for fodder, and leaf litter. Thus, we expected that the condition of forests in the plots is affected by household collection of forest products. We collected biomass data on major components of the forest carbon pool including tree and sapling biomass, leaf litter, grass and herbs. Using those data, we estimated total forest carbon per hectare. The four ecological variables recorded are binary indicators of forest fires, encroachment for farming, soil erosion and wildlife, and thus reflect forest management as well as ecological health. Vegetation type, altitude, slope, aspect and soil depth were also recorded for each plot.
In parallel, we undertook a multi-stage random sample survey of 15 households in each of 42 CFUGs in 2011 and 2013. Our final sample has a total of 1228 observations in 42 CFUGs, because 16 households could not be interviewed in 2013. These 16 missing households were from all three districts and spread across control and treatment CFUGs and therefore not related to location or treatment status. The household survey elicited socioeconomic information on livelihoods, harvest of forest products, and cooking technologies and fuels. The forest products are firewood used primarily for household cooking, grass as fodder for livestock, and leaf litter for animal bedding. Community characteristics were also recorded. Further details on the data are provided in the supplementary information (table S3).

Econometric model
Our BACI research design allows us to use the DID estimator to evaluate the effects of the REDD+ intervention on three sets of outcomes (Y i ), summarized as follows: (a) Intermediate socio-economic outcomes, measured at the household level, are expected to contribute to forest health and therefore forest carbon, as well as leading to co-benefits for households. Socio-economic indicators are quantities of firewood, fodder grass and leaf litter collected, cooking with an improved cook stove (ICS) or biogas, and the share of cooking done with firewood. (b) Intermediate ecological outcomes, measured at the plot level, are expected to contribute to forest carbon sequestration, as well as biodiversity co-benefits. These indicators are signs of forest fires, wildlife, soil erosion, and forest encroachment for agriculture or settlements. (c) Final forest carbon outcomes, measured at the plot level, include above ground biomass in four pools (herb, leaf litter, saplings and trees) as well as the total estimated aboveground and belowground carbon.
We estimated the following equation: where Y i is the vector of outcomes of interest; Treat is an indicator for the treated (REDD+) communities, Year is an indicator of the endline survey, and Treat * Year is an interaction of these two indicator variables. The stochastic error term is ε i . The coefficient on Treat * Year is the DID estimator of the impact of the REDD+ intervention (i.e. the intention-totreat effect) measured in units reported in table 1. Table 1 also reports means of the outcomes at baseline in the control CFUGs, for comparison with the size of the estimated impact.
Covariates are the two community characteristics that remained unbalanced post-matching-(i) the number of NGOs in a community, and (ii) the percent of households in a community using LPG for cooking at baseline. Further, forest carbon sequestration is a function of biomass growth, which depends on the biophysical characteristics of forested land (Yang et al 2006, Måren et al 2015. Therefore, we also include altitude, aspect, slope, and soil depth as covariates in our DID estimations on forest health and carbon stock.

Findings
For each outcome, table 1 summarizes (i) the DID estimate, (ii) the unit in which it is measured, and (iii) the mean of the baseline (2011) data from control CFUGs. The DID (treatment effect) estimates are displayed in figures 3-5. Note, these DID estimates suggest that REDD+ had fewer effects than simple comparisons of treatment plots before and after the REDD+ intervention (a common basis for offset credits) or tests for differences in the means of REDD+ and controls units after the intervention, as shown in the SI table S4. The SI (tables S5-S7) also provides the full estimation results for the DID models Figure 3 shows that REDD+ had no effect on the total quantities of firewood, fodder grass and leaf litter extracted by households. Focusing on firewood (which has the most impact on carbon), table 1 shows that while REDD+ did not significantly reduce the amount of firewood collected by households, it reduced the share of cooking with firewood by 5 percentage points. The results also suggest (at a 94% confidence level) that REDD+ increased the number of households adopting biogas, also coincidentally, by 5 percentage points. These two results suggest that REDD+ increased households using biogas from 9 to 14 percentage points and decreased the share of cooking done with firewood from 87.5 to 82.5 percentage points, Both are plausible estimates in the context of the averages in non-REDD+ villages in 2011 (table S3) and in REDD+ villages in 2013 (table S4).
We also examine the impact of the REDD+ pilot program on forest health by estimating equation (1) for four key indicators of human-nature interactions measured at the plot level. Figure 4 shows that the REDD+ project effectively increased the likelihood of wildlife, while decreasing the likelihood of forest fire, encroachment, and soil erosion. The largest effect is on forest fire, which is mostly anthropogenic in Nepal (Poudel et al 2014). REDD+ reduced the probability of forest fire in a plot by 38%. Additionally, the probabilities of forest encroachment and soil erosion declined by 23% and 18%, respectively, while the probability of observing signs of wildlife increased by 20%.
Total forest carbon stock is the sum of carbon in herbs, leaf litter, saplings, trees and estimated belowground carbon. As figure 5 shows, REDD+ had a statistically significant effect only on litter biomass (+1.4 tons per hectare). This is arguably one of the biomass pools that could change most rapidly in response to management, e.g. reduced collection of leaf litter and prevention of forest fires. However, the increase in leaf litter could also be related to the marginally insignificant decline in herb biomass, because increased litter cover prevents sunlight from reaching the ground, inhibiting growth of herbs.

Discussion
REDD+ is viewed as an important means to mitigate climate change by reducing net forest carbon emissions. When REDD+ prevents deforestation, it can be attributed with an immediate reduction in emissions. On the other hand, when REDD+ prevents degradation and enhances forest carbon stocks, it may not be possible to measure the impacts for years. Given the immediate demand for information about REDD+, including both its effectiveness at reducing emissions and its implications for the livelihoods of forest-reliant people, it is important to identify and evaluate impacts on predictive proxy indicators for eventual carbon and livelihood outcomes. When REDD+ is community based and demand driven, these intermediate outcomes depend on the choices made by communities.
Our study makes two major contributions: (1) we examine a community demand driven REDD+ pilot in which the communities that received payments chose to allocate them partly to changing household energy use; and (2) we assess impacts on both socioeconomic and ecological indicators of biomass energy use that could lead to reductions in carbon emissions. Thus, we generate evidence on how REDD+ simultaneously affects socioeconomic, ecological, and carbon outcomes. Most of the early research on avoided deforestation strikingly missed socio-economic outcomes (Caplow et al 2011), while much of the recent research on REDD+ has not evaluated carbon outcomes (Duchelle et al 2018), thus limiting our understanding of potential trade-offs and complementarities. Our study design, with a prematched sample and baseline data from households and forest plots, allows for a credible assessment of impacts across these domains.
While the ultimate impacts of forest conservation interventions cannot be observed or evaluated until years after the interventions, there is an urgent demand for information on REDD+ (Duchelle et al 2018). In our study, we demonstrate how to overcome this contradiction by examining short-term signals, including behavior changes reported by households and reflected in ecological indicators, which are likely to move carbon stocks and local well-being in the right direction. We find evidence of incremental progress based on our evaluation of intermediate outcomes. While follow-up studies can verify whether these intermediate outcomes ultimately lead to reductions in net carbon emissions, short-term studies are important given the urgent need for climate action (Ripple et al 2017).
Our findings indicate that there is cause for optimism regarding REDD+ success in the short term, with implications for the long term. Results, consistent with findings from Maraseni et al (2014), suggest positive impacts on key ecological indicators in treated communities, which are indicative of longer-term reductions in net forest carbon emissions. For example, we find a decline in forest fires and forest encroachment, cross-validated by improved signs of wildlife. While the estimated effects on socioeconomic indicators have the expected signs, they are generally not statistically significant, except for a 5% reduction in the share of cooking done with firewood.     This lack of impacts on socioeconomic indicators shows that improvements in ecological indicators do not have to come at the expense of immediate reductions in local forest benefits. We find no evidence that REDD+ threatens people's livelihood needs, such as fuel wood use, contradicting results from earlier work (Maraseni et al 2014), and allaying prevailing fears that prioritizing forest carbon sequestration through REDD+ in Nepal will undermine livelihood benefits (Poudel et al 2014, Luintel et al 2017. However, we acknowledge that our results reflect impacts in the short-term (over two years) and on the average household. Further analysis is needed to identify any effects on poorer households within CFUGs, and longer-term REDD+ pilots are needed to identify the effects of sustained implementation.
From a climate mitigation perspective, the lack of clear evidence of reductions in firewood extraction is problematic because firewood contributes to declining forest carbon in Nepal (Baral et al 2012, CBS 2019. The limited impact of REDD+ on household firewood collection may be because households stack (rather than 'switch') energy technologies Pattanayak 2012, Ruiz-Mercado andMasera 2015). While REDD+ did not reduce total firewood collected (for energy uses) over the time frame of the study, it reduced the share of firewood used for cooking, perhaps because households expanded their energy use by adding biogas plants. Because traditional cook stoves provide heating as well as cooking services (CBS 2019), households may have continued use of firewood for heating, even while they added biogas for cooking (Nepal et al 2020). These findings point to both the potential and the challenge of shifting household cooking and heating patterns to reduce net carbon emissions in Nepal, where more than 65% of rural households use firewood to cook (Nepal et al 2007, Ministry of Health 2017, CBS 2019. The DID estimation results show very limited impacts on forest carbon, with the only significant impact being an increase in litter biomass. One reason is the short duration of the study. Another reason could be that the REDD+ pilot project placed only a small weight (16%) on carbon increments in defining payments to the CFUGs (Sharma et al 2017). Further, there was no decline in forest carbon even for control CFUGs, indicating that communities were already sustainably harvesting their forests, with or without REDD+ . In order to have an impact on forest carbon pools, REDD+ would need to increase forest carbon pools, which again, would have been challenging to achieve in the short time span of this study.
The DID estimation with pre-matched data identifies impacts that are quite different from the conclusions that would have been drawn by tracking plots and households in the treated CFUGs (a common approach for estimating the offset credits generated by REDD+ projects) or by testing for differences in treated and control CFUGs after the REDD+ intervention (perhaps the most common way to evaluate impacts). This is because DID controls for both common underlying trends and for otherwise unobservable differences across CFUGs, as long as those are time invariant. In addition, observable differences in CFUGs are accounted for either in the matching process or as covariates. We matched on eleven characteristics of CFUGs and included two of those as covariates, along with basic biophysical characteristics of the plots.
If there are other unobserved and time-varying confounders (i.e. trends that vary systematically across REDD+ and control CFUGs and that are related to the socioeconomic, ecological, and carbon outcomes), those could offer rival explanations for our results. However, CFUGs did not apply or choose to participate in the REDD+ pilot, but rather became participants if they were located in a watershed selected by the project administration. Thus any unobserved time-varying confounders would have to be factors that influenced both administrative selection of watersheds and the socioeconomic, ecological or carbon outcomes. We do not think that there are any such confounders that are likely to be influential enough to change our basic results, although we cannot rule that out.

Conclusions
REDD+ has the potential to mitigate climate change relatively cheaply, while offering co-benefits to forestrich but economically underdeveloped countries Wertz-Kanounnikoff 2008, 2009). This potential has not been realized, both because of insufficient flow of funds for implementing REDD+ (Sunderlin et al 2015) and insufficient evidence on the impacts of REDD+ implementation (Duchelle et al 2018). Our study shows that a community based and demand driven approach to REDD+ can improve ecological indicators without constraining local livelihoods. While we verify progress, we also identify challenges, including the need to ensure that energy interventions effectively meet household needs and enable reductions in firewood collection. The design of both interventions and monitoring systems should account for the possibility that households may seek to expand their portfolio of fuels and their total energy use in order to meet multiple and increasing household energy needs. Finally, we observe positive impacts on ecological health, which suggest incremental progress towards reducing net forest carbon emissions, supported by our finding of a positive effect on one of the pools of forest carbon stock that can change in the short-run. This implies that monitoring and crediting systems, as well as impact evaluations, should consider these types of intermediate outcomes to better understand and predict the consequences of REDD+.

Data availability statement
The data that support the findings of this study are available upon reasonable request from the authors.A,