Assessing improvements in socio-ecological system governance using mixed methods and the quality governance framework and its diagnostic capacity tool

Abstract

We face extreme and unprecedented socio-ecological systems (SES) governance challenges given advances in technology, global biophysical change, human behavior, and our abilities to seek resilience through policy decisions. The associated demands to rapidly adapt and shift our trajectories for improved SES outcomes provide a great impetus for humans to improve upon complex SES governance quality. Building on theories derived from polycentric, participatory, network-based practices, structured deliberative decision processes, and capacities that align with resilient systems thinking, a Quality Governance Framework (QGF) and Diagnostic Capacity Tool (DCT) were developed to diagnose SES governance quality that could improve outcomes for a cleanup and redevelopment program in Michigan. The QGF and its DCT were found to be reliable and valid. Using a subset of the DCT measurements and other respondent data, this research uses a mixed methods approach to further test and validate the QGF and DCT. The cleanup program was in its third year of transitioning from a more hierarchical form of governance to one that is more participatory. The results indicate further convergence in ratings between previously disparate practitioner populations and increases in the perceptions of improved governance quality and resilient SES outcomes. Respondents’ open-ended feedback and program metrics correlate with the DCT findings and provide further validity for the QGF and competencies associated with quality governance measured with the DCT. This research contributes to the growing body of empirical evidence that can assess governance quality and illuminates how governance quality can be diagnosed to treat and improve SES outcomes.

Appendix: Open-ended question comments

1.1 Summary of respondent comments

1.1.1 Example comments

Improved governance

I believe the MDEQ is working hard to improve its relationship with the regulated community and is working productively with stakeholders. (external respondent)

I feel that the RRD has made strides in seeking stakeholder involvement and collaboration early on in processes especially compared to how this has been done historically. (internal respondent)

Improved governance; requires additional support

The culture within MDEQ has improved significantly over the past 12–18 months and most (not all) staff have embraced the change. Pockets of staff in senior roles remain that have resisted the change. (external respondent)

Insufficient governance development

Bias is given to external stakeholders and internal stakeholders are not included or given equal weight in decision-making. Politics are considered too often and guide department decisions to the detriment of our environment. (internal respondent)

The review process seems to still largely determined by the biases (good or bad) of the individual project manager.

(external respondent)

Insufficient governance development and/or perceived dissension or values conflict

The politics seem to be getting in the way of the accomplishments. (internal respondent)

Too many staff have reverted to the most conservative interpretation of the rules. Guidance is extremely conservative.

Gray areas are not acknowledged or discussed; there is only one interpretation, and that’s the particular staff person involved. (external respondent).

Perceived dissension or values conflict

Environmental responsibility has seriously eroded in the last 15 years, and state government increasingly allows business interests to dominate. (internal respondent)

Division management needs to use its employee resources more instead of looking to outside resources to determine the division policies and future. (internal respondent)

Internal respondents
1	The definition of stakeholders doesn’t seem to include the public, it seems to assume stakeholders are employees of the regulated community/consultants and are directly affected by RRD implementation of Part 201. The public and the natural resources of our state should be our primary stakeholders. Recently proposed amendments to Part 201 would further erode and make a mockery of #7, protecting our environment	Dissension or values conflict
2	I believe the practitioners in the Remediation and Redevelopment Division strive to offer excellent customer service and are happy to work with stakeholders on a way to get to “yes.”	Improved governance
3	Sometimes (often?) the opinions of external stakeholders carry more weight than the combined input & opinions of internal RRD (project managers, TAPS team members, etc.)	Insufficient governance
4	I feel that the RRD has made strides in seeking stakeholder involvement and collaboration early on in processes especially compared to how this has been done historically	Improved governance
5	Environmental responsibility has seriously eroded in the last 15 years, and state government increasingly allows business interests to dominate	Dissension or values conflict
6	Although the department is making strides in empowering the right people there remains a strong undercurrent of individuals who are covertly and overtly impeding progress. These individuals believe that the changes are counter our mission and believe we must be “more regulatory” and less helpful. This is counterproductive and creates internal conflict which is hurting local morale	Improved governance; requires additional support
7	The current group of stakeholders does not include all, or even a representative group, of relevant stakeholders. The current stakeholders group is definitely biased toward the economic side of things and not balanced with the social and environmental aspects	Dissension or values conflict
8	Division management needs to use its employee resources more instead of looking to outside resources to determine the division policies and future	Dissension or values conflict
9	RRD as a division, and staff individually, are working hard to manage environmental and natural resource problems while taking into account fully the need to balance solutions with socio-economic demands. We whole-heartedly support and work to facilitate the redevelopment of environmentally-challenged properties while ensuring that the risks are reliably managed in order to protect public health and the environment. While it appears that attitudes of many stakeholders and some members of the legislature are starting to recognize RRD’s efforts, there is too much uncertainty regarding long term funding needed to sustain these efforts and RRD programs in general	Improved governance; requires additional support
10	Government is not a business. We need to deliver true equity and objective, fact based, legal decisions. We seem to be acting more like a bunch of attorneys or business men who think objective truth is not critical: That the law is the law and not just a whim of management. I think this reflects a Governor who only understands business and has no appreciation for the principles of government. This shallow understanding of government will eventually result in an unethical governmental work force and I am already seeing the Department and Division creeping in this direction. “If the king listens to liars, eventually everyone in his court is a liar.” That is where we are headed. It is not Bob Wager’s or Susan Erickson’s fault. The creep started with Engler, but this Governor lacks the depth and experience to comprehend the damage that is being done	Dissension or values conflict
11	N/A	N/A
12	The “new” attitude of the DEQ-RRD is refreshing. Some staff in upper levels (decision makers) have not embraced the changes. Also, the DEQ = RRD continues to not seek out the staff most knowledgeable about issues for policy development and TAPS teams. They use those who are available or who can talk the best line. RRD continues to not respect the technical staff, their knowledge, education or abilities. Unless you want to be a manager this place is a dead end job and will have problems retaining people with the best technical abilities	Improved governance; requires additional support
13	Regarding an early question - staff WOULD embrace empowerment, if it were truly effectively in place	Insufficient governance
14	I did not answer Q2 because I don’t know what a “wicked” problem is as it relates to environmental concerns. The author should look up the word wicked in the dictionary…	N/A
15	I believe that these questions are chosen and phrased in a way that targets strong points without really leaving room for clarification. Clarification of the reasoning behind the answers would also serve as feedback for consideration in making improvements. Prioritization remains a problem in both lack of staffing and determination of funding allocation. Sometimes the big picture is too vague for and other times the focus is too small to consider pertinent information. While this is an expected hurdle in any organization, the department has had a steady increase in financial responsibilities in the past 20 years while funding has decreased steadily. This continues to the extent that each staff member makes prioritization decisions every day due to vacancies; these vacancies remain for as long as 10 years. The overall outcome is difficult to adequately convey when overload becomes the normal and low priority sites fall off the radar in some areas—yet other programs are allocated funding for tasks that haven’t even been balanced against them	Insufficient governance
16	Setting goals for the division is necessary, but setting too many goals all at the same time is not good for morale and is counterproductive, especially when the goals are being set during our busiest time of the year, i.e., during triage and action plans	Insufficient governance
17	I find it alarming when protecting the environment is item 7 on the Governor’s plan, when it is the same environment that so much of Michigan’s economy is built around. Having a strong environmental protection program, and not just giving it lip service, is a key component to Michigan’s overall economic recovery and should be given a higher priority	Insufficient governance and values conflict
18	Some of the questions were too theoretical in nature. Why not ask more questions being more specific?	Insufficient governance and values conflict
19	As a tax payer and citizen of Michigan, I am disappointed that DEQ staff are silent (as they have been instructed to be) as environmental regulations are rolled back, dismantled, and altered by law makers to allow more contamination to remain in the environment. As a citizen, I always thought DEQ staff were looking out for the general public, waters of the state, and future generations. However, now as a DEQ employee, I understand that my previous perception was incorrect	Insufficient governance and values conflict
20	Rather than using multiple spreadsheets and weekly meetings to track metrics, timesheets should be used (as in the private sector) to keep track of what staff are working on	N/A
21	To many people are spending the majority of their time on TAPS Teams and are not keeping up with their assigned site	Insufficient governance and value conflict
22	The politics seem to be getting in the way of the accomplishments	Insufficient governance and values conflict
23	It seems that position of the internal stewards on particular sites is often trumped by the external stakeholders at upper management levels, i.e., upper management does not take into consideration the salient issues of the site developed by the staff, but instead makes deals based on what the stakeholders would like to see as an outcome. There are backroom deals being made, and the decisions are largely based on economic considerations of the stakeholders and not on what is in the best interest of the resource for future generations of Michiganders	Insufficient governance support
24	bias is given to external stakeholders and internal stakeholders are not included or given equal weight in decision making. Politics are considered too often and guide department decisions to the detriment of our environment	Insufficient governance support
25	In the past few years, I’ve witnessed a notable improvement in the ‘quality’ of the RRD	Improved governance
26	Staff are overworked with competing priorities, including triage, compliance assistance, private party required responses (NFAs, RAPs, FARs), state funded sites (SS/AP) and committees. Way too stressful!!!!!!!!	Insufficient governance support
27	Management does not embrace, support or realize the importance of defining and collecting quality data to resolve environmental problems. Because management does not realize the importance, staff are not trained in how to collect quality data or properly evaluate data collection plans. Therefore, staff are making decisions with insufficient data or the wrong type of data with little or no understanding or consideration of confidence, representativeness or reproducibility	Insufficient governance support
28	More time is spent in planning and reaching consensus than actually working	Insufficient governance support Values conflict
29	With over 30 years in this business, I am tired of having to break in new governors/directors, etc., every 4-8 years and their own “programs”. Environmental cleanup and resources need a secure, consistent funding program that is not tied to a new governor/director’s interests. Environmental issues can take years/decades to complete and cannot be held hostage by short term politicians	Insufficient governance support
30	Regarding Q4, the jury is still out on meeting future demands. RRD is flexible and provides alternatives to complex environmental problems with the understanding that people will use adequate due care and will address a problem if it arises. Will the responsible parties and governmental representatives understand this and do the right thing if there is an environmental risk that needs to be addressed? Will the legislators provide monetary and legislative support to ensure that RRD can protect the public and the environment? RRD resources are stretched to the maximum	Insufficient governance support
31	I think RRD is doing a great job. People are engaged and exercising great customer service	Improved governance

External respondents
1	Culture within MDEQ has improved significantly over the past 12-18 months and most (not all) staff have embraced the change. Pockets of staff in senior roles remain that have resisted the change	Improved governance; requires additional support
2	The isolated, closed-minded and risk-averse culture at the staff level is challenging. Many staff members through the years have complained about the lack of empowerment, however most will also seek input from others in their office to gain a consensus opinion instead of using their own judgement. This leads to very conservative and ingrained decisions. Management continues to accept past behaviors and support such decisions by staff even though it seems inconsistent with what they say to other stakeholders	Insufficient governance
3	It was difficult to characterize my overall experience with RRD for several of the previous items as other than neutral, because of one experience which was very positive and one which was very much not so. (each with a different part of RRD: one which seemed to grasp changes to the regulations and embraced new authority to act on the given case based on what the circumstances of that care really warranted; the other which showed no grasp of the new statutory provisions and an inability to embrace and employ any new approaches to that case.)	Improved governance; requires additional support
4	I believe the MDEQ is working hard to improve its relationship with the regulated community and is working productively with stakeholders	Improved governance
5	DEQ RD do not agree between districts as to interpretation of rules. Also, DEQ staff seem to quote internal documents that are not available to public, but yet we are held accountable to them. DEQ staff are also somewhat reluctant to make an opinion, especially in writing	Insufficient governance
6	Overall I’ve experienced significant improvement in the way the department operates. Some “old school” views remain and there is room for improvement, but the future seems much brighter now than it did prior to Director Wyant’s arrival	Improved governance; requires additional support
7	Many senior RRD staff members appear to be purposely obstructing progress in remediating and closing sites by creating new obstacles to closure/NFA. It is very frustrating to invest significant resources into the remediation and closure of a facility only to have RRD staff (who will admit that they do not agree with the new risk based initiatives) engage in creative interpretations of the statute to invent new barriers to NFA status. The success of the 201 program is largely dependent on the willingness of the regulated community to invest in remediation. If RRD staff continue to invent impediments to NFA status then the regulated community will abandon its commitment of resources to advance remediation projects. This will bring you back to the dark days of having sites that will only be remediated when they are forced to be remediated through litigation or enforcement. Does RRD staff really have the resources to litigate or go through escalated enforcement on every case? Of course not, but that is the world you create by not providing a predictable path to closure without unnecessary administrative impediments. Simply put, if you put up enough roadblocks to predictable closure the regulated community will stop investing in cleanups and the entire process fails, harming the public and the environment	Insufficient governance and values conflict
8	I observe steady improvement in the mission to serve customers (in the broadest sense). I hope it continues, regardless of how administrations may change in the future. “Old thinking” is being replaced with views of the bigger picture and more constructive efforts. Protection of the environment is always a requirement for the Department’s decision making, but some of the past overkill and poor decisions made for “administrative convenience” are becoming less frequent (but it is still present)	Improved governance; requires additional support
9	Too many staff have reverted to the most conservative interpretation of rules. Guidance is extremely conservation. Gray areas are not acknowledged or discussed; there is only one interpretation, and that’s the particular staff person involved. There is still way too much inconsistency among offices, with the SE Michigan office the most inflexible, conservative, dictatorial, and service-averse	Insufficient governance and values conflict
10	The Department has gotten much much better to work with. Still some unusual requests or lack of quick turn around. But it is better	Improved governance; requires additional support
11	MDEQ RRD staff have been helpful in understanding conditions of the site and any incentives to redevelop. They are team-players are a part of the community	Improved governance
12	The review process seems to still largely determined by the biases (good or bad) of the individual project manager	Insufficient governance
13	All documents for FOIA should be made available via web, email or scanned format (as Grand Rapids district does). This is far more timely and cost effective for those seeking the information. Alternative approaches to Closure of LUST sites should be considered and more readily available	N/A
14	Answers to the previous questions were difficult because some staff and managers are willing/eager to explore solutions while others default to reasons against solving a problem in a socio-economic-health and environment balanced way. There are personalities that are obstructive to cooperating on a solution - closed-minded, and those that embrace and stretch to find a way to balance a solution for the greater good. Uncooperative staff and managers must be identified and managed according to their strengths and weaknesses as a great team can be wrecked by a couple players that have skills but don’t want to use them for the greater good	Improved governance; requires support