The Financial Return from Measuring Impact

In conservation, as in most other subjects, there is a division of expenditure into problem identification, solution testing, and practice. However, research concentrates on problem identification rather than solution testing. We calculate the return on the investment of research (a PhD thesis) examining the effectiveness of conservation interventions for birds of prey in three European countries. We show that the economic return from investing in a PhD thesis could be substantial, in the order of hundreds of thousands euros over 10 years or a return on investment of between 292% and 326% over that period. We derived the values of return on investment by first setting a common biological target (the total number of raptor fledglings produced per year). We then compared overall costs in achieving such target via the wide implementation of the results from the thesis (i.e., allocating resources to the most effective intervention) versus a business as usual scenario. We identify other theses that also show considerable benefits in improving effectiveness. We suggest that further research examining effectiveness would be cost‐effective in improving practice.


Introduction
Conservation biology has been defined as a crisis discipline requiring threats to biodiversity and ecosystems to be identified and tackled with urgency (Soulé 1985). Considerable efforts have thus been made to identify new (Sutherland & Woodroof 2009) anthropogenic threats. At the same time, large efforts have also been made to study existing threats and drivers of environmental change, from global to local scale issues, and bring them to the attention of practitioners, decision-makers, and the public (Laurance et al. 2012).
Meanwhile, a growing range of solutions has been identified that could help address those threats to biodiversity (Sutherland et al. 2015). However, there is an emphasis on problem identification to the detriment of solution scanning and evaluation (Ferraro & Pattanayak 2006). A lack of evaluations of conservation interventions is pervasive in conservation biology, and it is a major source of inefficient spending (Ferraro & Pattanayak 2006). It is also a major reason why the discipline of conservation biology lags behind other disciplines, like medical science, which develop through the practice of robust experimental testing of effectiveness of interventions and systematic review of the evidence (Sutherland et al. 2004).
Those interested in improving practice in any field, whether medicine or nature conservation, have the problem of allocating expenditure into three areas: the identification of problems, finding solutions to these problems, and carrying out practice (Runco 1994;Klein 2012;Pullin et al. 2013). In this article, we concentrate on the costeffectiveness of research on interventions.
Allocating more resources to evaluate the effectiveness of interventions may have several positive consequences. First, it may allow cutting unnecessary waste and inefficiencies in the spending of scarce resources, while having a positive impact on biological systems (Ferraro & well as encouraging practitioners, citizen scientists, and conservation scientists to maintain or increase their commitment in conservation (Jones 2012). Evaluating the effectiveness of interventions is nowadays more readily achievable than in the past given a relatively long history of implementation of various interventions and associated accumulation of high-quality monitoring data (Ferraro & Pattanayak 2006). At the same time, quantitative methods for carrying out robust evaluations are constantly improving (e.g., Howe & Milner-Gulland 2012a).
It can be argued that there is a considerable need for more research devoted to evaluating the effectiveness of conservation interventions (Sutherland et al. 2004) in order to reduce the imbalance between action and threat identification, and evaluating interventions. However, it can also be argued that we need action rather than more research. The question then is whether funding the assessment of evidence is an effective investment through making practice more cost-effective. Clearly, the relevance to applied conservation of research conducted in academia is much lower than that conducted within Nongovernmental organizations and governmental agencies (Smith et al. 2009). Academia has thus a great potential for increasing the number of evaluation studies and aid the accumulation of evidence in support of conservation decisions. This can be achieved by using part of the large workforce represented by postgraduate students. Here, we provide a case study where we calculate the return on investment resulting from the wide application of the results of a PhD thesis focused on assessing the effectiveness of conservation interventions . We then discuss the implications that targeted postgraduate work could have to improve the cost-effectiveness of conservation.

Economic return from investing in a PhD thesis
We calculated the potential monetary savings associated with the application of the results of a PhD thesis in conservation biology (extended methods are presented in supporting material Appendix S1). The thesis ) focused on assessing the effectiveness of alternative interventions for protecting raptor nests in man-managed landscapes.
The PhD thesis was based on five studies, but here we consider only those three where interventions required monetary investment. These three studies considered protection of Montagu's harrier nests Circus pygargus in farmland of France (Santangeli et al. 2015), a separate study of this species in Spain (Santangeli et al. 2014), and protection of nests of the white-tailed eagle Haliaeetus albicilla in Finnish forests ).

Quantification of costs for each intervention
For the two studies in France and Spain, the costs for each implemented intervention were given in the respective publications. In Spain, costs were quantified by determining expenses for the additional fieldwork necessary to apply the protection measure and the direct implementation costs, such as compensation to a farmer for delaying harvest; see more details in Santangeli et al. (2014). Intervention costs in Spain may vary according to nest aggregation and crop productivity. This variation was captured by quantifying the minimum and maximum costs for each intervention (Table 1). In France, the fieldwork is carried out by volunteers and there is no compensation to farmers for applying any protection measure. The only expenditures included are for building a small fence to protect from predation (applicable to two interventions, see Table 1). This cost is in the order of 5-10 euros according to estimates from the year 2015 (Santangeli et al. 2015).
For the white-tailed eagle study, only aggregated costs for protection across Finland for each year starting from 1996 till 2010 were available. We thus derived the protection costs per eagle nest by dividing the average national yearly investment in nest protection in Finland by an estimated number of nests that were protected per year (extrapolated to the country level from more accurate information available for SW Finland). In each case (this applies only to the French and Spanish study), we applied a conservative approach by considering the minimum and maximum cost estimates for each protection intervention (Table 1). For the Finnish case, only an average was available (but variation in costs was assessed with sensitivity analyses; see Appendix S1).

Quantification of effectiveness of each intervention
Effectiveness of each protection measure was quantified, within each study, by comparing the modeled productivity (i.e., the number of fledglings per nest; see respective studies: Santangeli et al. ( , 2014Santangeli et al. ( , 2015 of each intervention against the productivity of unprotected nests in each study system. The added benefit of each intervention was derived by making the difference between productivity from protected and unprotected nests. Ultimately, the most effective intervention identified among those available in each study was the one yielding the largest added benefit. For Spain and France, removal or relocation and fenced relocation were the most effective   In the case of Spain, National population size is taken considering only populations in the following administrative areas where nest protection applies: Lleida, Madrid, Castilla La Mancha, Extremadura, Andalucia, Castilla y Leon, and La Rioja. Note: The different rows depict values for a specific intervention implemented in a given year for a given species in a given study (MH, Montagu's harrier and WTE, White-tailed eagle). Extra fledgling productivity per nest depicts the surplus of fledglings from protected nests over the reference value from unprotected nests. Protection costs per nest are yearly additional (see methods) costs for protecting a nest (Min. and Max. depict minimum and maximum cost estimates where available from the original studies). National population size depicts the overall number of pairs of a species in each country, whereas the next column to the right depicts the % of nests protected using each intervention over the total population of that country. Next, the number of nests protected with each intervention, as well as the total costs for protecting those nests (again given as minimum and maximum) and the total benefits (total n. of extra fledglings produced from protecting the specific number of a nests with the specific intervention) are given separately for the Business As Usual (i.e., according to the current allocation of nests to be protected with each intervention) and for the Thesis scenario (whereby in each of the three countries, nest protection is implemented by using only the most effective intervention; i.e., the one with the highest extra fledgling productivity per nest). Number of protected nests under the Thesis scenario is adjusted so as to reach a common (between BAU and Thesis scenario) target of 3,384 extra fledglings overall. The sum of total costs (minimum and maximum values) and the benefits (i.e., the common biological target) for each scenario are shown at the bottom right of the table. All costs are euros, whereas all benefits are in n. of extra fledglings Figure 1 The costs and gains from each intervention included in this case study. The six interventions for protecting Montagu's harrier nests in France are in white, the three interventions in Spain are in black, and the intervention to protect white-tailed sea-eagle nests in Finland is in gray. The size of each dot is proportional to the percentage of nests protected with each intervention across each respective country (see Table 1). Negative values result from interventions that reduce fledglings' production.
interventions, respectively. In the Finnish case, no benefit was found from nest protection , thus the added benefit of this intervention was considered as zero. Next, we use the effectiveness of each intervention to derive two contrasting scenarios: one that we call Business as usual (hereafter BAU) where allocation of conservation effort among alternative interventions will continue as currently done, and one called the Thesis scenario whereby conservation effort is concentrated only on the most effective intervention per study as determined by the PhD thesis (see above and Table 1). When costs and gains (here, we consider costs per nest protected and gain of extra-fledglings resulting from nest protection) from each intervention are compared, it becomes clear that there is a large variation in the cost versus gain ratio (see Table 1 and Figure 1). Some interventions are very cheap but yield high gains, whereas other interventions are relatively expensive but yield no apparent gain. This finding further highlights the need to evaluate the cost-effectiveness of conservation interventions, so that resources are allocated to those interventions that maximize the gains for a given cost, or minimize the costs to achieve a set biological goal (see below).

Quantification of the overall costs and benefits of the BAU and Thesis scenario
We then estimated current expenditures considering the national populations of the studied species. Data from France considered protection at the national level (Santangeli et al. 2015). In that country, we derived the proportion of nests protected with each intervention over the total number of nests from a large database including over 6,000 nests from the period 2007 to 2012 (Santangeli et al. 2015). The total national population of breeding Montagu's harriers in France was obtained from (Thiollay & Bretagnolle 2004).
In Spain, the study included in the PhD thesis (Santangeli et al. 2014) considered only part of the area, but this included a large proportion of the national breeding population of the Montagu's harrier (Arroyo & García 2007). We obtained a number of protected nests under each intervention across the whole country (where nest protection applies, i.e., excluding areas with populations breeding in natural vegetation) from the proportion of protected nests under each intervention over the total number of nests in each relevant area (Arroyo & García 2007;National Harrier Meeting 2015).
We then calculated the aggregated costs for protecting nests with each intervention per year, and also the aggregated benefits (total extra fledglings produced) per intervention and year across all three studies (see Table 1). This was done separately for the BAU and the Thesis scenario.
In order to compare the BAU and Thesis scenario, we set a common biological target, i.e., the number of extra fledglings that would be produced under the wide application of the BAU scenario (n = 3,384 extra fledglings overall in the three studies). We then calculated the costs necessary to achieve such a target with the BAU and with the Thesis scenario. This was done by simply summing the costs of protecting all nests with a specific intervention necessary to reach the common target of 3,384 extra fledglings per year. We calculated the summed maximum and minimum costs for each scenario, which were used for further calculations of the return on investment (see below and Table 1).
We obtained costs for a PhD thesis of 4 years duration at a Finnish University (including salary and overheads), this was estimated to be approximately € 156,211 in 2013. In practice, this is often an overestimation, because many PhD theses in Finland are funded via grants provided by external private foundations, which typically amount to under € 100,000 for a full 4-year PhD thesis.

Calculation of net present value savings and return on investment
We derived the net present value of the savings (i.e., the difference in overall costs between BAU and Thesis scenario). Finally, we calculated the return on the investment (ROI) from the overall costs for completing a PhD thesis (hereafter thesis costs) and savings over the following 10 years (hereafter NPVsavings10) discounted using a 5% rate (Groom et al. 2006) using the following formula:

ROI = [(NPVsavings10 -thesis costs) / thesis costs]
* 100 see, e.g., Jeffery (2004). We present below the ROI calculated based on the maximum and minimum cost estimates for each protection intervention. The rationale for using the 10-year period in calculating the ROI is that this can represent a reasonable time span when the effectiveness of the interventions could be assumed to be applicable given changes in landscape and conservation context. Longer time spans would in fact increase uncertainty. The net present value savings resulting from the wide application of the PhD thesis results (i.e., the difference in costs when achieving the common biological target between the Business As Usual and the Thesis scenario) over 10 years ranged between € 612,768 and 665,894 (using, respectively, the maximum and minimum cost estimates and a 5% discount rate). Given the initial investment of € 156, 211 for a PhD thesis, the return over a 10-year period ranges between 292% and 326%.
We would, however, caution that the above results include some uncertainty owing to limitations in the design of the original studies, as well as assumptions on projected impacts of the interventions as well as cost estimations. All of the original studies are based on observational empirical designs that may suffer from biases (Ferraro & Pattanayak 2006) regarding the decision on which nests to protect. These potential biases have been discussed in the original studies and are believed to have little impact on the overall results of this work, as also supported by our sensitivity analyses where variation in costs and nest productivity are assessed ( Figure S1). However, we emphasize the need and importance of having a solid experimental design in order to be able to quantify the causal effects of interventions (Ferraro & Pattanayak 2006) and ultimately derive robust return on investment measures.

Implications from increasing research on measuring impact
The results of this case study on the return on investment of a PhD thesis demonstrate the potentially high savings for conservation from investing in PhD theses testing the effectiveness of interventions. This is mostly due to the large variation in the cost-effectiveness of different implemented interventions (see Figure 1). As a consequence, there are large opportunities for redistributing resources toward the most cost-effective interventions.
Academic conservation science often does not seek to resolve applied on-the-ground conservation problems (Braunisch et al. 2012). Similarly, evaluation studies with robust designs and high relevance to applied conservation are rare (Ferraro & Pattanayak 2006). The workforce of postgraduate students could revolutionize global conservation by engaging in evaluation studies and ultimately leads to more effective practice.
We recognize that challenges exist to increase the proportion of postgraduate thesis evaluating intervention effectiveness. The performance of academics is heavily reliant on journal impact factors instead of being assessed, at least within the field of conservation, on the relevance of their science to practical applications (Arlettaz et al. 2010). There is a general perception that evaluation studies rarely lead to publications in highranked journals (although see Hoffmann et al. [2010]), so they may appear unattractive to researchers including supervisors and students. However, evaluation studies should be highly attractive to governments, private foundations, NGOs, and other funders. Solutions may include universities responding to the calls for greater relevance and impact, practitioners paying for research, and practitioners also carrying out the required research themselves.
The conclusion of this article, that testing effectiveness may yield high returns on the investment, could be a consequence of one highly atypical thesis. We, however, believe that this may not necessarily be the case. Although few theses do test interventions, we can list other examples. Sarah Eglington's PhD thesis showed the costs and benefits, to waders and farmers, of alternative interventions for water management in grasslands (Eglington et al. 2008;Eglington et al. 2010). Inês Catry's PhD thesis assessed the effectiveness of an intervention (the provision of nest boxes) to increase populations of an endangered farmland raptor (Catry et al. 2009). Malcolm Ausden's PhD thesis investigated the effects of interventions, such as winter flooding of wet grasslands, on the prey base available for waders with important implications for conservation of this endangered group of birds (Ausden et al. 2001). Jake Bicknell's PhD thesis explored the consequences of improving timber harvest, via the reduced-impact logging intervention, for tropical biodiversity, providing evidence that the intervention can improve the biodiversity sustainability value of timber harvest across vast tropical regions (Bicknell et al. 2014). Anna Berthinussen's PhD thesis (Berthinussen & Altringham 2012) showed that bat gantries (overhead structures assumed to guide bats in safely crossing roads) were ineffective, but underpasses had potential to allow bats to cross roads safely in the UK. In Finland, the PhD thesis of Juha Pöyry showed no effect of restorative cattle grazing on the community of butterflies and moths in seminatural grasslands (Pöyry et al. 2004).
Ultimately, the balance between investing in conservation actions versus evaluations may be strongly dictated by the manner in which interventions are implemented.
As stated earlier, many conservation interventions implemented worldwide may suffer from weak designs (Ferraro & Pattanayak 2006). Evaluation studies using the resulting monitoring data from such weak study designs may produce weak and potentially biased inferences. This may have negative repercussions for achieving effective nature conservation. In these cases, investing in planning and executing a solid experimental study design for implementing conservation interventions may represent a better option than relying on inferences from weakly designed interventions.

Conclusions
There are massive conservation challenges to address (Hoffmann et al. 2010), a pervasive lack of resources, large inefficiencies in conservation spending (Ferraro & Pattanayak 2006), and poor representation of scientific knowledge at the environmental policy level (Dicks et al. 2014). All these issues are, to some extent, related to the lack of scientific evidence supporting appropriate decision-making. At the same time, countless PhD and Masters theses are produced every year in the field of ecology and biology. If we could channel even a small proportion of such postgraduate students' workforce toward assessing the effectiveness of conservation interventions and generating evidence, this could result in large savings for conservation. It will also generate a positive feedback whereby more evaluation studies are produced every year as the students may address such topic also in their future career. Moreover, the high return on the investment may attract increasing amount of resources as the number of evaluations grows. Ultimately, and most importantly, it will allow learning from past actions and addressing current and future conservation challenges more effectively, thereby increasing the chances of achieving conservation goals.