Effectiveness of community forest management at reducing deforestation in Madagascar

Community Forest Management (CFM) is a widespread conservation approach in the tropics. It is also promoted as a means by which payment for ecosystem services schemes can be implemented. However, evidence on its performance is weak. We investigated the effectiveness of CFM at reducing deforestation from 2000 to 2010 in Madagascar. To control for factors confounding impact estimates, we used statistical matching. We also contrasted the effects of CFM by whether commercial use of forest resources is allowed or not. We cannot detect an effect, on average, of CFM compared to no CFM, even when we restricted the sample to only where information suggests effective CFM implementation on the ground. Likewise, we cannot detect an effect of CFM where commercial use of natural resources is allowed. However, we can detect a reduction in deforestation in CFM that does not permit commercial uses, compared to no CFM or CFM allowing commercial uses. Our ﬁndings suggest that CFM and commercial use of forest resources are not guarantees of forest conservation and that differentiating among types of CFM is important. (cid:2) 2015 The Authors. Published by Elsevier Ltd. ThisisanopenaccessarticleundertheCCBYlicense(http:// creativecommons.org/licenses/by/4.0/).


Introduction
The major role of tropical forests in biodiversity and climate change has led the world to search for effective ways to slow deforestation.Many approaches have come in and out of fashion.Strictly protected areas, which prohibit most human activities, were popular in the early days of conservation and remain so today.As an alternative to strict protected areas, Community Forest Management (CFM) emerged in the late 1980s (Hutton et al., 2005).By virtue of involving local forest users in management, CFM is promoted as having the potential to benefit both forests and local livelihoods (Behera, 2009).This potential, however, has been questioned (Behera, 2009) and its evidence base has been found to be weak (Bowler et al., 2012).Although Payments for Ecosystem Services (PES) have become the most recent fashion in efforts to reduce deforestation, CFM remains an important part of the forest management toolkit in many developing countries (Blaikie, 2006).It is also promoted as a means by which PES schemes can be implemented.High quality studies evaluating the effectiveness of CFM are therefore important for shaping future development and investment in approaches to reduce deforestation.We aim to provide robust evidence on effectiveness of CFM at reducing deforestation.
Studies investigating the effectiveness of conservation interventions often fail to adequately control for confounding factors that affect both the assignment of interventions and the outcomes of interest (Bowler et al., 2012;Ferraro and Pattanayak, 2006;Joppa and Pfaff, 2010).Recent studies investigating the effectiveness of protected areas at reducing deforestation have made progress in controlling for confounding factors by the use of statistical matching (e.g.Andam et al., 2013Andam et al., , 2008;;Carranza et al., 2014;Ferraro et al., 2013).Matching selects comparison areas that have pre-intervention baseline values of confounding factors most similar to intervention area values, and thus makes it possible to control for these confounding factors (Joppa and Pfaff, 2011).However, we know of only one study (Somanathan et al., 2009) that has used matching to investigate the effectiveness of CFM at reducing deforestation.
A significant challenge for evaluating the effectiveness of CFM is the large variation in forest management practices and designs within the approach, both among and within countries (Lund et al., 2009).In terms of practices, examples of this variation range from cases where the community has a good understanding of their rights and responsibilities to cases where CFM exists on paper only (Benjamin, 2008;Lund et al., 2009).An example of design variation is that some CFMs allow communities to benefit from commercial use of forest resources within their managed forests while others do not (Persha et al., 2011).Failure to consider this variation compromises the potential for learning about design and implementation factors that promote CFM effectiveness.
The Malagasy government legislated CFM in the late 1990s (Raik, 2007) to reduce deforestation and protect the significant part of the world's biodiversity that is endemic to Madagascar (Le Saout et al., 2013).The number of CFM units increased rapidly and continues to grow (Aubert et al., 2013).Many publications review the institutional and political aspects of Madagascar's forest decentralization process (Pollini et al., 2014;Pollini and Lassoie, 2011;Raik and Decker, 2007;Rives et al., 2013;Urech et al., 2013), but only a few focus on empirically estimating the performance of CFM in terms of conservation outcomes (CIRAD, 2013;Sommerville et al., 2010;Toillier et al., 2011).None adequately control for factors that may confound impact estimates.
Using statistical matching to control for factors that confound impact estimates, we investigate the effectiveness of Madagascar CFM at reducing deforestation between 2000 and 2010.To our knowledge, this is the first national scale study of performance of CFM at delivering conservation outcomes.First, we assess the overall effectiveness of Madagascar's forest decentralization policy at reducing deforestation by looking at all CFM units across the country.Second, we distinguish and study effectiveness in a subsample of CFM units where we have information to suggest that CFM was implemented on the ground.Finally, we differentiate between CFM that allows and does not allow commercial use of forest resources and study effectiveness conditional on whether CFM permits or prohibits commercial use.Note that we do not consider other important potential outcomes from CFM including impacts, positive or negative, on human welfare.

Study areas
In Madagascar, the transfer of forest management to local communities involves three main steps; the creation of a local forest management group, adoption of forest rules, and signed contract between the local forest management group, the state forest department and possibly the municipality where the forests are located (Aubert et al., 2013;Pollini and Lassoie, 2011).In our study, CFM refers to forests managed by communities that achieved these three steps.
Our study covers CFM established between 2000 and 2005 (Fig. 1).Because 2010 is the end of the period of our analyses, selecting CFM established between 2000 and 2005 allows observing at least five years of deforestation impacts post CFM establishment.There is no national database containing current information on all CFM units.We therefore gathered information from multiple sources including organizations involved in implementation; namely, Direction Générale des Forêts, Office National pour l'Environnement, Asity, Fanamby, Durrell Wildlife Conservation Trust, Conservation International, Wildlife Conservation Society, and World Wide Fund for Nature.
Malagasy CFM varies in their implementation quality.Some were established with little input from local communities (Rives et al., 2013), and others received little or no external support (Hockley and Andriamarovololona, 2007).It is very difficult to get information of the implementation quality of the individual CFM unit.We used whether a CFM unit passed the forest department evaluation that is undertaken three years after the contract (Pollini and Lassoie, 2011) as an indicator of whether the project was indeed implemented.While not an ideal indicator, it does at least suggest the CFM unit has met the basic institutional, socio-economic and environmental criteria of the evaluation.We refer to units that passed the evaluation as CFM units that have information to suggest implementation.
CFM implementation in Madagascar varies according to regulations related to commercial use of forest resources.Commercial CFM allows commercial uses and adopts it as a conservation strategy.Non-commercial CFM does not permit commercial uses and follows a pure conservation strategy (Randrianarivelo et al., 2012).Because there are no reliable national data regarding where commercial uses are permitted within CFM, we conducted analyses on commercial and non-commercial CFM for four sites only, where we were able to ascertain information on commercial uses through field visits, interviews with site managers or search of existing literature.The four sites are Didy, Tsitongambarika, Menabe and Boeny (Fig. 1).All CFM units that we considered in these four sites had passed the forest department evaluation.Table 1 presents the number of CFM units, the area of land and natural forest covered by each type of CFM considered in our analyses.
Non-CFM areas refer to forests that, up to 2010, were not technically and financially supported by particular organizations and thus were under government control.Since the government has been weak and unable to enforce forest laws, these forests are subject to open access (Raik, 2007;Urech et al., 2013) We excluded six out of the 22 administrative regions of Madagascar where we were unable to collect CFM data (Fig. 1).Because we analyzed CFM established between 2000 and 2005, undated CFM and CFM established before 2000 or after 2005 were excluded.We also excluded protected areas managed by Madagascar National Parks.Finally, extensions of protected areas, temporary and new protected areas created since 2003 were excluded.However, any portions of these newly created protected areas that were known to be community managed were considered as CFM (Fig. 1, see Appendix D Table D1 for how CFM, non-CFM and excluded areas fit into official Madagascar forest statuses since 2003).

Matching, unit of analysis, sampling
Conservation interventions like CFM are not randomly assigned.The site characteristics that affect where conservation interventions are assigned also affect deforestation, thus confounding attempts to estimate intervention impacts (Ferraro and Pattanayak, 2006).To control for these confounding factors, some empirical studies have used matching (Andam et al., 2008;Joppa and Pfaff, 2011).Matching selects comparison areas that are similar to the intervention areas in terms of their values of the confounding factors at the pre-intervention baseline.Thus, one assumes that the outcomes of the comparison group represent, in expectation, the counterfactual outcomes of the intervention sites had they not been exposed to the conservation intervention.
The unit of analysis is a forested pixel from the 2000 forest cover baseline (See Appendix A for limitations of using 2000 baseline forest cover and CFM established between 2000 and 2005, and Appendix B for how we deal with potential pseudo-replication in which pixels within a particular CFM are not independent).For each forested pixel at baseline, covariates take the values of each confounding characteristic at that pixel location.For each analysis (Table 2), we selected random forested pixels in intervention areas.Then, we used matching to pair each randomly selected pixel with the most similar pixel in comparison areas in terms of covariates.The outcome variable is whether a pixel remained forested or not in the 2010 land cover.The estimated difference in deforestation between intervention and similar comparison areas represents the impact of the intervention on deforestation for intervention sites or the Average Treatment effect on the Treated (ATT).We used independent samples T-test to compare deforestation in intervention and similar comparison or counterfactual areas.
Our study comprises six analyses (Table 2).The first analysis compares all CFM established between 2000 and 2005 to non-CFM.The second compares the CFM that has information that suggests implementation to non-CFM.The four remaining analyses regard commercial and non-commercial CFM.Ferraro et al. (2013) demonstrated that four ATTs are policy relevant for studies involving two types of intervention.The third and fourth analyses compare commercial and non-commercial CFM to non-CFM.The fifth and sixth ones compare commercial to non-commercial CFM, and vice versa.The difference between these two latter analyses rests upon the type of the sampled pixels used in the comparison.The CFM forests where commercial use is permitted may be observably different from the CFM forests where such use is prohibited (in terms of the confounding factors).To understand how deforestation in commercial CFM forests would have been different without commercial use requires that we compare commercial CFM forests to non-commercial CFM forests that are observably similar at baseline (in terms of the confounding factors).So the comparison of commercial to non-commercial CFM uses an intervention group of all the randomly selected commercial CFM pixels and a comparison group of only the best matches of non-commercial CFM pixels.Dissimilar pixels from the non-commercial CFM sample are discarded.The estimate from this comparison represents the average impact of permitting commercial use on the types of CFM forests where commercial use has been permitted.The impact of commercial use may be different on the types of CFM forests where commercial use has been prohibited.To estimate this impact, we formed an intervention group of all the randomly selected non-commercial CFM pixels and a comparison group of only the best matches of commercial CFM pixels.
In all analyses, we used Mahalanobis covariate matching because it better balances covariates than other matching options.We performed exact matching on vegetation zones (eastern humid, western deciduous and southern spiny forests, Appendix E Fig. E1).We executed bias adjustment regression to correct for any remaining post-matching covariate imbalance (Abadie and Imbens, 2006).We used the ''matching'' package in R (Sekhon, 2011).
We aimed to select sample sizes that balance our interests in achieving high statistical power and reducing computer processing time.Learning from multiple trial analyses, we decided on a sample of around 30,000 pixels for all intervention areas in each analysis.For comparison areas, we sampled around two to four times more pixels (Appendix D Table D2).The larger sample size from comparison areas increases the probability of finding a good match for each intervention pixel.

Forest cover
We used 2000 and 2010 deforestation data developed by ONE et al. (2013).These are based on images from Landsat TM and Landsat ETM+ and have a resolution of 28.5 m and an accuracy rate close to 90%.Full methods are in Harper et al. (2007).

Covariates or confounding baseline characteristics
Based on Madagascar CFM practitioners' opinion, and CFM and deforestation studies in Madagascar and other tropical countries (Barsimantov and Kendall, 2012;Bowler et al., 2012;Forrest et al., 2008;Gorenflo et al., 2011;Sussman et al., 1994), we identified pressure and access as potentially confounding factors.To control for these factors, we used measures of agricultural suitability, slope, elevation, distance to recent deforestation (1990)(1991)(1992)(1993)(1994)(1995)(1996)(1997)(1998)(1999)(2000), distance to forest edge, distance to a village, distance to an urban center, distance to a road, distance to a cart track, duration of trip to an urban center and population density (see Appendix D Tables D3, D4 for sources of covariate data).Because community characteristics received little consideration in selection of community forests for CFM designation, we did not consider community characteristics as confounding factors but only condition on these site characteristics indicating pressure and access (see Appendix C).

Sensitivity analysis to unobservable bias
While matching can ensure that the distributions of observable covariates are similar between intervention and comparison groups, the groups may still differ in terms of unobserved covariates that affect both deforestation and assignment to intervention.To check the robustness of our estimates of effectiveness to such unobservable covariates, we performed Rosembaum's (2010) sensitivity test.A parameter C measures the dissimilarity in the likelihood of receiving intervention between intervention and counterfactual units due to unobservable covariates.In the absence of unobservable differences, C takes the value of one 1.The higher the value of C, the more dissimilar is the likelihood of receiving intervention for the matched pair due to unobserved variables.
The sensitivity analysis consists of increasing the values of C and determining a critical C at which the estimate of effect of intervention is not significantly different from zero.In other words, we seek to measure how strong an unobservable confounder would have to be in order for the estimated effect not to be significantly different from zero.The higher the value of the critical C, the more robust is the estimate of intervention effect to unobservable bias.We carry out sensitivity tests with the ''rbounds'' package in R (Keele, 2010).

Results
Before matching, CFM pixels are, on average, located closer to recent deforestation, to a road and to an urban center and are characterized by shorter trip durations to an urban center than non-CFM pixels.Although these patterns suggest CFM is assigned to areas of higher deforestation pressure, CFM is also located on lands less suitable for agriculture and at higher elevation (Appendix D Table D5).
Commercial CFM pixels are, on average, associated with lands more suitable for irrigated rice, closer to a village, a road and an urban center, shorter trip duration to urban center, higher population density, but they are located on lands less suitable for agriculture and on steeper slopes than non-commercial CFM sites (Appendix D Tables D9, D10).
Matching generally improves covariate balance.The mean differences and the mean raw eQQ differences of covariates in intervention and counterfactual areas tend toward zero after matching (Appendix D Tables D5-D10).An exception is suitability for agriculture in the comparisons of commercial and non-commercial CFM, and vice versa.Matching does not improve balance for this factor (Appendix D Tables D9, D10).This is because all suitable lands for agriculture are found only in the non-commercial CFM in Menabe.Thus, there are no matched suitable lands in commercial CFM.We describe potential effects of this imbalance in the discussion.
Between 2000 and 2010, CFM sites had, on average, 0.02% less deforestation than matched non-CFM sites, a statistically insignificant difference (p = 0.99, Fig. 2).When we consider only CFM with information suggesting implementation, CFM had 0.76% less deforestation than matched non-CFM, but still statistically insignificant (p = 0.71).Differentiating CFM by whether commercial uses are allowed, we estimate that commercial CFM experienced 1.83% more deforestation than matched non-CFM (p = 0.16).Non-commercial CFMs reduced deforestation by 2.01% relative to matched non-CFM (p < 0.001).When we compare commercial CFM to matched non-commercial CFM, to investigate their relative effectiveness on the types of CFM forests where commercial use has been permitted (i.e., forests on lands more suitable for irrigated rice, closer to a village, a road and an urban center, shorter trip duration to urban center, higher population density), commercial CFM experienced 3.24% more deforestation (p < 0.001).Comparing non-commercial CFM to matched commercial CFM, to investigate their relative effectiveness on the types of forests where commercial use has been prohibited (i.e., forests on lands less suitable for irrigated rice, farther to a village, a road and an urban center, longer trip duration to urban center, lower population density), we estimate non-commercial CFM reduced deforestation by 5.59% (p < 0.001, Fig. 2).
Table 3 presents the results of the sensitivity of our analyses to hidden bias (i.e., an unobservable covariate).For example, where the parameter C is 1.38, the estimate of 2.01% remains significantly different from zero at a p value of 0.05 even if an unobservable covariate makes non-commercial CFM pixels 1.38 times more likely to receive intervention than non-CFM pixels.In other words, unobservable covariates need to increase the likelihood of the non-commercial pixels to receiving intervention by a factor greater than 1.38 in order for the impact estimate not to be statistically different from zero.

Discussion
Decentralization of forest management to local communities in Madagascar has not, on average, achieved its forest conservation goal.In terms of deforestation, we cannot detect an effect, on average, of CFM compared to no CFM, even after restricting the sample to only where we have information to suggest CFM implementation on the ground.
Many studies report success and failure of CFM at delivering conservation outcomes (Cox et al., 2010;Pagdee et al., 2006;Porter-Bolland et al., 2012).However, in a systematic review of CFM performance in developing countries, Bowler et al. (2012) showed that evidence on effectiveness of CFM is weak because of poor study design.They proposed a ''gold standard'' that would produce quality CFM assessment.We believe that our study meets the ''gold standard'' as far as is possible (though only for a single potential outcome from CFM -that of reducing deforestation).That is, we use comparator sites, baseline forest cover data, multiple CFM across Madagascar and paired or matched design.We also sample randomly the unit of analysis and allow enough time (5-10 years) for impacts to take place.Finally, we identify and control for confounding factors that may bias impact estimates.
While CFM failed, on average, to reduce deforestation relative to non-CFM, non-commercial CFM appears to have had more success, albeit a small one.Putting all types of CFM in one basket would lead to the single conclusion that CFM is not an effective approach to reduce deforestation, obscuring the positive impact noncommercial CFM appears to have had.This result emphasizes the importance of differentiating among types of CFM in evaluation (Lund et al., 2009).Potential mechanisms through which noncommercial CFM may have had more success are complementary direct payments for conservation.Some non-commercial CFM in our study sites in Didy, Tsitongambarika and Menabe practice a direct payment for conservation scheme to offset restrictions introduced by interventions (e.g.Brimont and Bidaud, 2014;Sommerville et al., 2010).
The estimated reduction in deforestation from non-commercial CFM is important given that the role of commercial use of forests in conservation is a subject of much debate in theoretical and empirical studies.Some studies argue that by assigning value to forests, commercial use provides means and incentives to local communities to protect forests, while others show that it can trigger the destruction of the resources being commercialized (Agrawal and Chhatre, 2006;Barsimantov and Kendall, 2012;Persha et al., 2011).Our findings do not support the argument that permitting commercial extraction can enhance the deforestation-reducing impacts of CFM.
Our matching algorithm was unable to remove the pre-matching difference between commercial and non-commercial CFM in terms of agriculture suitability (Appendix D Tables D9, D10).After matching, commercial CFM has lower suitable lands for agriculture (0%) than non-commercial CFM (29%) has.However, knowledge of the direction of the effect of agriculture suitability on deforestation allows us to infer the implications of the post-matching imbalance.Gorenflo et al. (2011) show that lower suitability for agriculture is associated with lower deforestation rates in Madagascar.Thus, the post-matching imbalance should occasion lower deforestation in commercial CFM than non-commercial CFM.Therefore, if matching had balanced the suitability for agriculture between the two types of CFM, commercial CFM performance relative to non-commercial CFM would have appeared even worse because the lower deforestation occasioned by the lower land suitability in commercial CFM would have been erased.Our estimates of impacts for commercial vs. non-commercial CFM, and vice versa are thus conservative.
At the national level, our findings substantiate the rather gloomy pictures of CFM in Madagascar depicted in a number of institutional and policy studies (Pollini et al., 2014;Pollini and Lassoie, 2011;Raik and Decker, 2007;Rives et al., 2013;Urech et al., 2013).To explain the ineffectiveness of CFM, these studies describe inadequate integration of local participation, resource capture by elites, unfulfilled support promises by different organizations, and lack of capacity of the community and state, among other factors.While these studies point to institutional and policy shortcomings, we advance our understanding of CFM performance by empirically showing that at the national scale, there was no impact in terms of delivering a central objective: reducing deforestation.A recent empirical study (CIRAD, 2013) is of particular interest because it also looked explicitly at the impact of CFM on deforestation and covered part of our study areas, and its results contradict ours.It found that deforestation was significantly less in CFM than in areas without community conservation.It also shows that commercial CFM was more effective at reducing deforestation than non-commercial CFM.The results are not directly comparable to ours because the analyses cover a different time period and are at a different spatial scale, but the CIRAD study should be interpreted with care because they did not adequately control for the biases in confounding factors that we do here.Failure to adequately control for such biases can result in incorrect impact estimates (Andam et al., 2013(Andam et al., , 2008;;Joppa and Pfaff, 2011).
Many factors can influence effectiveness of CFM (Agrawal, 2003).We focused on the potential role of commercial use of forest resources (and given our study area, the potential that complementary direct payment for conservation could have in non-commercial CFM).Another potential moderating factor is the amount of resources invested, which may explain the apparently better performance of non-commercial CFM relative to commercial CFM we observe.During our visit to Didy, commercial CFM officials complained about receiving smaller resources relative to their neighboring non-commercial CFM (implemented by different organizations with different funding).However, we lacked the quantitative information on spending to allow this potential moderator of success to be included in the analysis.Data on this moderator in the future will also offer opportunities to extend our study by exploring CFM cost-effectiveness.
We focused on comparing CFM to non-CFM and different types of CFM.Other studies attempt to compare CFM to strictly protected areas (IUCN categories I-IV) to investigate the relative effectiveness of these two different approaches (Bray et al., 2008;Porter-Bolland et al., 2012).We do not attempt such a comparison because we believe that investigation of the relative effectiveness of strictly protected areas and CFM at reducing deforestation cannot be done credibly for Madagascar with the same robust and rigorous methodology we used here.Matching covariates and baseline forest cover ideally should be measured before intervention (Andam et al., 2008).CFM units in Madagascar were established in or after 2000 while 80% of the strictly protected areas were created before 1970.Thus, strictly protected areas and CFM have different baselines (2000 vs. pre-1970).If these baseline measures are to be used, the start time for the analyses will have to be different for CFM (2000) and strictly protected areas (pre-1970).Therefore, the comparisons of CFM to strictly protected areas, and vice versa, are difficult for Madagascar because the impacts will be estimated for dramatically different time periods.In addition, communities around and managers of strict protected areas and those of CFM have different length of experience in exposure to intervention that may explain the difference of impacts between Fig. 2. Differences in percent deforestation between intervention and counterfactual (#CFM where we have information to suggest implementation, ⁄ significant at p < 0.001, error bars: standard errors for post-matching estimates that are calculated using a variance formula that is robust to heteroskedasticity and adjusts the variance estimator for repeated matches among control units (Abadie and Imbens, 2006).In conclusion, we provide robust evidence that CFM and commercial use of forest resources are not guarantees of conservation success.Our findings also suggest that differentiating among types of CFM is important when evaluating effectiveness.By explicitly estimating impacts conditional on the type of CFM, scholars can shed light on the factors that promote effective CFM.
Contents lists available at ScienceDirectBiological Conservationj o u r n a l h o m e p a g e : w w w .e l s e v i e r .c o m / l o c a t e / b i o c o n

Fig. 1 .
Fig. 1.Map showing CFM established between 2000 and 2005, commercial and non-commercial CFM sites and areas excluded from the analyses (Projection: Laborde Madagascar).

Table 1
Number of units and dimension of different types of Community Forest Management (CFM).

Table 3
Sensitivity tests to unobservable covariates.Analysis Critical C at p = 0.05 Non commercial CFM vs. non CFM 1.38 Commercial CFM vs. non commercial CFM 1.50 Non commercial CFM vs. commercial CFM 5.85 the two approaches.Such comparisons may yield credible results in specific sites or in other countries, where strictly protected areas and CFM were established around the same time.