Meta-Analysis of Livelihood Impacts of Payments for Environmental Services Programmes in Developing Countries

Abstract

Payments for environmental services (PES) programmes have been widely promoted over the last few decades in many developing countries. Improving the livelihoods of environmental services (ES) suppliers is not only seen as a side benefit but is often considered a prerequisite for the viability of PES. Yet, the ability to draw ‘overview lessons’ over the impacts of PES on livelihoods from literature review studies remains limited. To overcome these shortcomings, we undertake a meta-analysis of causal statistical studies on the effects of PES on the livelihoods of ES suppliers in the developing world. The set-up of our meta-analysis allows us to draw more conservative but more reliable and generalisable overview lessons. Our findings suggest that PES programmes are likely to have positive but modest livelihood impacts on ES suppliers. Further, several institutional characteristics of PES are found to be correlated with more favourable livelihood impacts, such as high payments, high degree of voluntary participation, low transaction costs and better access to alternative income sources. Lastly, our results highlight the importance of controlling for unobservable confounders when undertaking original evaluation studies on the impacts of PES. These factors should be incorporated in the design, implementation and evaluation of PES.

Introduction

The last decade has witnessed a rapid growth of payments for environmental services (PES) programmes in developing countries (Ferraro et al., 2012; Pattanayak et al., 2010). Typically PES schemes entail a form of Coasean transaction in which environmental service (ES) users make payments to ES providers, in exchange for the provision of ES or pre-agreed actions intended to deliver ES (Engel, 2016; Wunder, 2015).¹ In principle a crucial pre-condition for the viability of PES schemes is that the benefits accruing to the suppliers of ES should exceed the costs of ES provision (Engel et al., 2008; Pearce, 2004). In other words, the net livelihood impact on payment recipients should be positive. This pre-condition is particularly crucial when PES schemes are applied in the developing world, as in this context these policies often have the dual function of addressing environmental goals and alleviating poverty (Wunder et al., 2008). It is, thus, important to assess and understand the magnitude, direction (positive or negative) and determinants of livelihood impact of PES polices in the developing world.

Yet, although there is a substantial body of empirical work that aims to assess the impact of various PES schemes, our ability to draw robust ‘big-picture’ or ‘generalisable’ lessons remains limited. Literature reviews or synthesis papers such as those by Grieg-Gran (2005), Jack (2008), Wunder (2008), Pattanayak (2010), Miteva (2012), Tacconi (2013), Alix-Garcia (2014), Engel (2016), Hejnowicz (2014), Samii (2014), Börner (2017), Ma (2017) and their co-authors are able to provide a qualitative assessment of the environmental and socioeconomic impacts of PES. Though their inferences over actual PES impact are informative, they remain very case specific. We are unaware of any robust quantitative syntheses of available empirical evidence (such as via meta-analysis) of the livelihood impact of PES. What has perhaps hindered this research agenda of deriving more overview lessons is that the possible universe of ‘empirical evidence’ is vast and quite heterogeneous. Empirical assessments on the impacts of PES studies use different forms of data and analytical methods at different levels of scientific rigour. Further, a large proportion of the empirical work on evaluating the outcomes of PES has the limitation of not being able to credibly trace the causal relationship between a PES and its observed ‘effects’, either on the affected communities or the environment (Blackman, 2013; Ferraro et al., 2012; Miteva et al., 2012). That is many empirical studies provide purely correlational findings and do not adequately control for confounding factors that could lead to observed impacts and which are unrelated to the PES itself (Ferraro and Pattanayak, 2006; Baylis et al., 2016). Combining empirical results from studies that use such varied methods and data into a meta-analysis would lead to results which would be unreliable and of limited informational value.

This paper addresses these challenges in deriving reliable overview lessons from the existing empirical body of work on PES by making two key contributions. First we perform perhaps the first meta-analysis of empirical studies that aims to explore the impact of PES programmes on the livelihoods of ES suppliers in developing countries. Our analysis adds to more qualitative past attempts that have relied mostly on literature reviews. Secondly, we include in our analysis only the evidence from ex post impact evaluation studies that use statistical methods that explicitly aim to assign a causal link between PES and observed outcomes. The quality of any meta-analysis is only as good as its inputs (Glass, 1976) so keeping with best practice guidelines we confine our analysis to include studies that entail a minimal level of methodological similarity.

In particular, we use studies that employ impact or programme evaluation methods that seek to identify the (treatment) effects of PES on livelihoods whilst controlling for and isolating from other confounding factors. These methods include experimental methods which control for confounders through randomly assigned treated and control groups, and quasi-experimental approaches such as matching, difference-in-differences, instrumental variable and regression discontinuity methods (Ferraro and Hanauer, 2014; Greenstone and Gayer, 2009; Imbens and Wooldridge, 2009; Khandker et al., 2010). The need for focusing more on such methods in assessing conservation policies (and PES in particular) has been widely and vigorously argued for in recent literature (e.g. Ferraro and Pattanayak, 2006; Greenstone and Gayer, 2009; Pattanayak et al., 2010; Miteva et al., 2012; Cowling, 2014; Baylis et al., 2016; Börner et al., 2017; Sills et al., 2017). Admittedly, impact evaluation methods are not without flaws nor are they the only scientifically credible approaches for assessing policies. Yet, for the purposes of undertaking a valid meta-analysis it is imperative to include studies that share a minimal common methodological ground. This reduces both the subjectivity in selecting the included studies but also the ‘noise’ in the raw data to be used. This rationale conforms with the ‘best evidence synthesis’ approach, as advocated by the seminal publications on meta-analysis by Hunter et al. (1982) and Slavin (1986) but also by best practice guidelines within economics (Stanley and Doucouliagos, 2012; Nelson and Kennedy, 2009). In the present meta-analysis, we focus on empirical studies that are explicitly geared towards addressing the issue of attribution via statistical methods. In doing so, we are better positioned to draw more reliable inferences about the causal relationships between PES programmes and key outcome variables. Our approach aims to maximise the possibility of producing clear and credible (albeit more conservative) overview lessons from the existing body of empirical work.

The quantitative nature of our study and the focus on causal statistical evidence distinguish it from other qualitative syntheses and literature reviews. Although the studies of Oltmer (2000), Brouwer (2011), Ezzine-de-Blas (2016) and their co-authors provide rare examples of quantitative syntheses assessing PES programmes, they do not focus on livelihood impacts nor do they limit their analysis to causal statistical studies. As such, the present study is able to draw more conservative but perhaps more reliable and more generalisable overview lessons on the likely impact of PES on livelihoods. Our analysis provides evidence on how to design and implement such programmes so as to enhance the livelihood impact and the long term viability of PES programmes. Further, our results shed light on the influences of different (causal) evaluation methods on the measurement of livelihood impact which provide lessons to those working on policy assessment of PES.

This paper proceeds as follows. Section 2 sets out our main hypotheses based on stylised facts on the livelihood impact of PES programmes as derived from past empirical studies and literature reviews. Section 3 describes our data used in the meta-analysis and Section 4 discusses our empirical strategy. The results are presented in Section 5, and the paper concludes in Section 6.