Towards quantitative condition assessment of biodiversity outcomes: Insights from Australian marine protected areas

Protected area management effectiveness (PAME) evaluation is increasingly undertaken to evaluate governance, assess conservation outcomes and inform evidence-based management of protected areas (PAs). Within PAME, quantitative approaches to assess biodiversity outcomes are now emerging, where biological monitoring data are directly assessed against quantitative (numerically de ﬁ ned) condition categories (termed quantitative condition assessments). However, more commonly qualitative condition assessments are employed in PAME, which use descriptive condition categories and are evaluated largely with expert judgement that can be subject to a range of biases, such as linguistic uncertainty and overcon ﬁ dence. Despite the bene ﬁ ts of increased transparency and repeatability of evaluations, quantitative condition assessments are rarely used in PAME. To understand why, we interviewed practitioners from all Australian marine protected area (MPA) networks, which have access to long-term biological monitoring data and are developing or conducting PAME evaluations. Our research revealed that there is a desire within management agencies to implement quantitative condition assessment of biodiversity outcomes in Australian MPAs. However, practitioners report many challenges in transitioning from un- dertaking qualitative to quantitative condition assessments of biodiversity outcomes, which are hampering progress. Challenges include a lack of agency capacity (staff numbers and money), knowledge gaps, and diminishing public and political support for PAs. We point to opportunities to target strategies that will assist agencies overcome these challenges, including new decision support tools, approaches to better ﬁ nance conservation efforts, and to promote more management relevant science. While a single solution is unlikely to achieve full evidence-based conservation, we suggest ways for agencies to target strategies and advance PAME evaluations toward best practice.


a b s t r a c t
Protected area management effectiveness (PAME) evaluation is increasingly undertaken to evaluate governance, assess conservation outcomes and inform evidence-based management of protected areas (PAs). Within PAME, quantitative approaches to assess biodiversity outcomes are now emerging, where biological monitoring data are directly assessed against quantitative (numerically defined) condition categories (termed quantitative condition assessments). However, more commonly qualitative condition assessments are employed in PAME, which use descriptive condition categories and are evaluated largely with expert judgement that can be subject to a range of biases, such as linguistic uncertainty and overconfidence. Despite the benefits of increased transparency and repeatability of evaluations, quantitative condition assessments are rarely used in PAME. To understand why, we interviewed practitioners from all Australian marine protected area (MPA) networks, which have access to long-term biological monitoring data and are developing or conducting PAME evaluations. Our research revealed that there is a desire within management agencies to implement quantitative condition assessment of biodiversity outcomes in Australian MPAs. However, practitioners report many challenges in transitioning from undertaking qualitative to quantitative condition assessments of biodiversity outcomes, which are hampering progress. Challenges include a lack of agency capacity (staff numbers and money), knowledge gaps, and diminishing public and political support for PAs. We point to opportunities to target strategies that will assist agencies overcome these challenges, including new decision support tools, approaches to better finance conservation efforts, and to promote more management relevant science. While a single solution is unlikely to achieve full evidence-based conservation, we suggest ways for agencies to target strategies and advance PAME evaluations toward best practice.

Introduction
As the establishment of protected areas (PA) continues to grow to meet international targets (CBD, 2011), conservation management agencies are tasked with evaluating their management effectiveness to ensure they achieve the best conservation outcomes (Coad et al., 2015). To assist conservation practitioners in doing this, the International Union for the Conservation of Nature (IUCN) recommended a six-step process to evaluate protected area management effectiveness (PAME; Hockings et al., 2006). PAME evaluation encourages the routine assessment of the entire management process, from documenting the PA management context and planning, accounting for the allocation of resources (inputs) and management actions undertaken (process and outputs), through to measuring conservation outcomes. By routinely assessing the entire management process, PAME evaluation promotes evidence-based conservation management of PAs, and the public reporting of PAME results helps provide transparency for reporting progress towards conservation objectives (Hockings et al., 2006;Leverington et al., 2010).
Since the release of the IUCN's PAME guidelines, PAME evaluation has been undertaken in more than 100 countries (Leverington et al., 2010). PAME evaluation methodologies involve assessments of indicators for each of the six elements of the management process undertaken by staff responsible for PA management (Coad et al., 2015;Leverington et al., 2010). Typically, PAME evaluation tools are questionnaire-based that require staff to use the best available evidence and their expert judgement to assess management effectiveness (e.g., CMP, 2013;Ervin, 2003;Stolton et al., 2007). For some aspects of management, qualitative data are most appropriate (e.g., assessing stakeholder engagement), but for other aspects (e.g., measuring ecological condition) quantitative data sourced from monitoring or research are considered most appropriate to support robust evaluations (Hockings et al., 2009).
Outcome assessments within PAME involve evaluating the condition of environmental (e.g., biological or physiochemical) indicators to determine whether conservation outcomes are being achieved or if management should be adapted (Hockings et al., 2006). This requires an assessment of environmental indicators (e.g., the abundance of a threatened species) against condition categories (e.g., "poor", "moderate" or "good"). In line with the assessments of other aspects of management, conservation outcomes are typically assessed using expert judgement, requiring staff to judge the condition of environmental indicators against generic statements of environmental condition, supported by available evidence (Fig. 1a; hereafter qualitative condition assessment).
Emerging approaches involve defining condition according to quantitative categories (e.g., poor condition is <10% coral cover), and thus enable the direct use of quantitative monitoring data to evaluate conservation outcomes (e.g., GHHP, 2016;Timko and Innes, 2009, Fig. 1b; hereafter quantitative condition assessment). These quantitative approaches reflect the more objective and statistically-based assessments of conservation outcomes used in the peer-reviewed literature (e.g., impact evaluation; Ferraro and Pattanayak, 2006;Coad et al., 2015). Quantitative condition assessments also address calls made by many scientists to better integrate environmental monitoring data into conservation evaluation and evidence-based management (Cook et al., 2016;Fox et al., 2014).
Both qualitative and quantitative condition assessments rely on clearly defined conservation objectives (e.g., the maintenance of biodiversity), and environmental indicators to assess whether objectives are being achieved (Hockings et al., 2006). However, the difference between these two assessment types lies in the way that: i) condition categories are defined (qualitatively or quantitatively); and, ii) the evaluation of environmental condition is undertaken, and the degree to which expert judgement is used (Fig. 1).
Qualitative condition assessments, using expert judgement, are necessary when limited or no monitoring data are available to inform the assessments of biodiversity outcomes (Cook et al., 2010). However, the use of expert judgement in environmental assessments can be subject to a range of biases, such as linguistic uncertainty and overconfidence (Burgass et al., 2017;Burgman et al., 2011). Experts have been demonstrated to interpret the scope, scale and timeframe of qualitative statements differently (Cook et al., 2014), and provide highly subjective estimates of environmental condition that can vary considerably between individuals (Burgman et al., 2011). The subjective nature of qualitative condition assessments means that the resultant accuracy and repeatability of PAME evaluations can be seriously compromised. Thus, PAME evaluations may have limited comparability through space and time.
Quantitative condition assessments are a viable alternative in situations where adequate scientific evidence is available (i.e., long- Fig. 1. Hypothetical examples of a) qualitative condition assessments, and b) quantitative condition assessments for a natural value (e.g., an indicator of biodiversity) within a protected area or network of protected areas. The key aspects of each approach are also described, including 1) how condition categories are defined, 2) how condition assessments are conducted, 3) the evidence used for assessments, and 4) how uncertainty is accounted for in condition assessments.