Comparing stakeholder perceptions with empirical outcomes from negotiated rulemaking policies: Is participant satisfaction a proxy for policy success?

Highlights

• Lack of robust, ecological data make it difficult to evaluate natural resource management policies.

• Natural resource policy evaluation may, thus, rely on stakeholder perception of success as a proxy for empirical success.

• We examined a long-term, ecological dataset as part of the Marine Mammal Protection Act’s Take Reduction Planning process.

• Stakeholder perceptions of policy outcomes were then compared with ecological outcomes.

• Stakeholder perception can be an accurate reflection of ecological outcomes, but not necessarily a predictor of them.

Abstract

Evaluation of natural resource management policies often is made difficult by lack of robust or long-term data on the resource. In the absence of empirical data, natural resource policy evaluation may rely on expert or stakeholder perception of success as a proxy, particularly in the context of policies that depend on multi-stakeholder engagement or negotiated rulemaking. However, few formal evaluations have compared empirical ecological outcomes with stakeholder perception. This study compares stakeholder perceptions of policy outcomes with ecological outcomes from a long-term, ecological dataset as part of the U.S. Marine Mammal Protection Act's Take Reduction Planning process. Structural Equation Models revealed that stakeholder perceptions were significantly and positively related to positive ecological outcomes. Also, perceived success and ecological performance rankings of the Take Reduction Plans were comparable for three of the five plans examined. This analysis suggests that for this particular policy instrument, stakeholder perception aligns well with ecological outcomes, and this positive relationship is likely the result of a commitment and support for stakeholder education and engagement. However, even within a single policy analysis, there was variability suggesting that the relationship between stakeholder perceptions and policy outcomes must continue to be evaluated. This study suggests that stakeholder perception can be an accurate reflection of ecological outcomes, but not necessarily a predictor of them.

Introduction

The U.S. federal government involves the public in regulation of natural resources along a continuum of engagement. At one end lies the command and control method wherein an administrative agency proposes regulations, releases them for public comment, modifies those rules in response, and implements final rules. At the other end of the continuum, stakeholders work directly with administrative agencies to devise regulations through consensus-based, multi-party negotiation, referred to as negotiated rulemaking [1], [2]. Various environmental agencies in the U.S. have embraced the latter approach, including the Environmental Protection Agency, Department of the Interior, and National Oceanic and Atmospheric Administration (NOAA) [3].

Assessing the efficacy of a policy in relation to program goals is fundamental to policy evaluation [4]. One critical metric of resource policy evaluation is whether the policy resulted in the intended goal, which is to improve resource condition, quality and quantity. However, for policies that are designed to protect natural resources, long-term resource monitoring data often are lacking. In lieu of direct data on the resource, other evaluations for environmental policies generated by multi-stakeholder programs may focus solely on the success of the negotiation process, while others focus on outputs or agreements resulting from the negotiation. Other evaluations focus on participant satisfaction with the process, which affects satisfaction with the outputs [5], [6]. Participant satisfaction, however, may not be a good measure, proxy, or indicator of successful ecological outcomes [3], [7], [8], [9], [10], [11]. Coglianese [9] points out that to avoid cognitive dissonance, stakeholders involved in intensive participatory processes such as negotiated rulemaking, may have a more positive view of the outcomes than is warranted by the outcomes themselves [5]. To date, few studies have considered how well stakeholder perceptions align with empirical trends [3], [9], [11], [12]. In other words, few studies have examined whether stakeholder perceptions of mission success or failure are accurate.

One negotiated rulemaking program administered by NOAA is mandated by the Marine Mammal Protection Act of 1972 (MMPA 16 U.S.C. 1361 et seq.). This negotiated rulemaking process, called Take Reduction Planning, develops plans that are designed to reduce harmful interactions between marine mammals and commercial fisheries (16 U.S.C. 1387).

The Take Reduction Planning program of the MMPA requires both long-term monitoring and negotiated rulemaking to mitigate the incidental capture of marine mammals in fisheries (bycatch). A recent study of the MMPA Take Reduction process found that the policy led to measurable empirical reductions in marine mammal bycatch, often referred to as takes [13]. To better understand the relationship between perceived and empirical ecological outcomes, this study quantitatively and qualitatively compares empirical ecological outcomes of marine mammal Take Reduction Plans [13] in relation to stakeholder's perceived outcomes [5]. This study serves to characterize the strength of the relationship between perceived and actual ecological success, directly informing the suitability of participant perceptions as a reliable proxy for empirical policy success.