Engaging in Enhancement : Implications of Participatory Approaches in Higher Education Quality Assurance

Stemming from increased levels of participation and diversity of the student base and from growing scrutiny on the quality of university degrees, governments have begun establishing mechanisms to monitor and support quality in higher education. Faced with administrative quality assurance structures, academics often complain that little is discussed in terms of collaborative effort toward program opportunities, enhancement, and growth and that focus is solely on accountability. This paper examines the viability of participatory approaches, pulled from the field of evaluation, as a way of leading to a more meaningful enhancement-oriented quality assurance process and creating common ground across the differing interests of external and internal stakeholders to quality in higher education.

igher education across the globe is experiencing a significant change in its student base.Increasing enrolments, greater diversity of the student body, and stronger links between school and workplace are being requested by those enrolling (Biggs & Tang, 2011).As a result, key stakeholders (such as students, parents, society, and governments) in higher education are taking a greater interest in the student experience and the quality of higher education (Dunn, McCarthy, Baker, & Halonen, 2010).With the increased profile of quality assurance, these stakeholders are calling upon specific government agencies and institutions of higher education to manage assurance systems that ensure standards are maintained and are accountable to professional accreditation bodies and public expectations (Kis, 2005;Nicholson, 2011).For many academic programs under review by quality assurance systems, processes related to this type of evaluation are perceived as an administrative hurdle imposed by distant senior administrators for the purposes of accountability (Kis, 2005;Nicholson, 2011).Academics complain that little is discussed in terms of collaborative effort toward program opportunities, enhancement, and growth (Harvey, 1998(Harvey, , 2002;;Nicholson, 2011).
Practitioners posit the viability of a system which can both be accountable to external stakeholders while engaging those internal to institutions (e.g., faculty, departmental chairs, and staff) to become invested in the sustained enhancement of their programs (Kis, 2005).Scholarship in the fields of quality assurance and evaluation suggest that a greater emphasis on collective participation in the design, administration, interpretation, and implementation phases of the assurance process could better engage internal stakeholders.Consequently, this could lead to a more meaningful enhancement-oriented process and create common ground across the differing interests of external and internal stakeholders (Cousins & Earl, 1995;Houston & Paewai, 2013;Kleijnen, Dolmans, Willems & Van Hout, 2013;Zukoski & Luluquisen, 2002).
To examine opportunities for an enhancement oriented participatory approach to quality assurance, this paper has been organised into two major sections.The first defines quality, describes the development of quality assurance frameworks, and discusses the nature of tensions between accountability and enhancement approaches to assurance.The second section presents the principles of participatory evaluation, highlights implications of use for quality assurance in higher education, and examines benefits and challenges of adopting a participatory approach.

Defining Quality in Higher Education
As it does in a general context, the meaning of the term quality varies considerably in the higher education context.Based on the influential work of Harvey and Green (1993) who examined the multiple meanings of quality, many scholars have emphasised five principal definitions of quality that are represented in higher education (Cheng 2014;Law, 2010;Nicholson, 2011).1) Exceptional -Linked to the idea of excellence, quality is operationalised as exceptionally high standards of academic achievement and is realized if the standards are surpassed.2) Perfection -Focuses on set specifications and standards that it aims to meet.Quality in this sense is summed up by the interrelated ideas of zero defects and getting things right the first time.3) Quality as fitness for purpose -a functional definition which suggests that quality only carries meaning in relation to the purpose of its product or service.A purpose which is generally characterised by an institutional mission or by customer (student) requirements.4) Quality as value for money -born out of a drive for efficiency and effectiveness, providers are expected to be accountable to funders (principally government) and customers (students).5) Quality as transformation -grounded in the notion of "qualitative change" (Harvey & Green, 1993, p. 24).This definition removes emphasis on product and service to consumer, instead seeing quality in education as an ongoing process of transformation of a learner which leads to both enhancement and empowerment (Harvey & Green, 1993).
Amid the various definitions, quality is seen as relative to the user of the term and of the situation in which it is being used (Burrows, Harvey & Green, 1992;Nicholson, 2011).For instance, for students and instructors, a view of quality may focus on the educational process, while for employers and government agencies it may focus more specifically on outputs.Perhaps, as mentioned by Harvey and Green (1993), rather than trying to define quality in a singular capacity the focus should be on defining the criteria used by the variety of stakeholders when judging quality and taking these into account in evaluative processes.

Approaches to Quality Assurance
Resulting from increases in levels of participation and diversity of the student base, coupled with pressures on human and physical resources, notions of quality began increasing in profile in the higher education agenda in the 1980s and 1990s (Biggs & Tang, 2011;Harvey & Green, 1993).Until that time, quality was largely determined by the number of faculty members with terminal degrees, the number of volumes in the library, reputation, size of endowment, etc., which were utilized in ranking systems (such as the Maclean's University Ranking) to determine institutional excellence (Koslowski, 2006).The increased profile of quality along with stronger links between quality and cost-effectiveness being made by the government, gave new urgency to the analysis of quality in higher education.This urgency led to the expedited establishment of quality assurance agencies which borrowed from existing approaches used in the corporate management arena, such as Total Quality Management and Continuous Quality Improvement (Koslowski, 2006).Embedded into the higher education context, quality assurance became performance oriented.As a result, and fueled by initiatives like the Bologna process in Europe, focus was placed on outcomes-based approaches that emphasized the identification and measurement of competencies that students should gain during their degrees (Nicholson, 2011).Ascribed to this approach, the most commonly accepted definition of quality from the late 1990s aligned with fitness for purpose (Harvey, 1998;Woodhouse, 1999).Nicholson (2011) credits this to the flexible nature of the outlook, as institutions could measure quality in terms of their ability to achieve the institutional objectives and mission.
Since education falls under provincial mandate, Ontario established the Ontario Universities Council on Quality Assurance (OUCQA) in 2010 (OUCQA, 2012).With targeted funding to get the initiative off the ground, the first order of business was to develop a series of Degree Level Expectations (DLEs) to identify the knowledge and skill competencies reflective of progressive levels of intellectual and creative development accomplished by a student at the end of a program.These DLEs were developed to benchmark the Ontario Quality Assurance Framework, all while providing individual programs the ability to differentiate themselves and define unique characteristics (OUCQA, 2012).As such, quality in the Ontario Quality Assurance Framework is predominantly operationalized using a fitness for purpose approach.For example, as quoted in Nicholson (2011), criteria used to evaluate both undergraduate and graduate programs include "consistency of the program with the institution's mission and academic plans" and "clarity and appropriateness of the program's requirements and associated learning outcomes in addressing the institution's own undergraduate and graduate Degree Level Expectations" (Quality Assurance Transition/Implementation Task Force and the Ontario Council of Academic Vice-Presidents' Executive Committee [QA Task Force], 2010, p. 8).Additionally, the DLEs measure fitness for purpose at the program level: The DLEs "combined with the expert judgment of external disciplinary scholars, provide the benchmarks for assessing a program's standards and quality" (QA Task Force, 2010, p. 18).
With quality assurance systems in Ontario and around the globe deriving from business and industry, discussions in academia regarding their applicability to higher education have been numerous (Nicholson, 2011;Owlia & Aspinwall, 1996).The most common elements raised relate to the customerbased focus of approaches from business and how this focus aligns poorly with the complex demands on higher education that go beyond a customer-supplier relationship, which is typically motivated by profit (Houston, 2008).In lieu of the fitness for purpose approach, academics commonly argue that good quality education is based on concern for the growth and transformation of the student (Cheng, 2014;Harvey, 1998Harvey, , 2002;;Nicholson, 2011).

Tensions between Accountability and Enhancement in Quality Assurance
Given the origins of quality assurance approaches and the complexities of stakeholders involved in higher education, tensions exist between approaches which focus on assurance and accountability and approaches which focus on student learning, growth and transformation, and enhancement of educational processes (Harvey & Williams, 2010;Hodson & Thomas, 2003;Kis, 2005;Law, 2010).Accountability is associated with a verification process which aptly renders account to external groups such as accreditation bodies, government agencies, and the public, while enhancement focuses on internal processes for the purposes of development and improvement.According to Koslowski (2006), the concept of "quality occupies the middle ground between the external and the internal; a philosophy or system that focuses and guides the interaction between the external calls for increased accountability and the internal efforts of an organization that is addressing it" (p.280).
Numerous scholars claim that accountability and enhancement are incompatible as the openness necessary for educational improvement is not part of the aim of accountability and that any formative benefits are likely to be incidental (Houston & Paewai, 2013;Kis, 2005;Newton, 2000;Nicholson, 2011;Vroeijenstijn, 1995;and Woodhouse, 1999).To examine this tension in greater detail, the characteristics that define accountability and enhancement approaches are summarized in Table 1.
As highlighted by Borden (2010), quality procedures for accountability toward external stakeholders engender a more summative and policy driven (top down) approach using typically standardised measures (traditionally quantitative) constructed for the purposes of strengthening external insight and control, broad comparison across higher education institutions and reporting to the general public.In contrast, quality assurance for internal enhancement involves a formative and faculty-driven (bottom-up) approach using a variety of measures (both quantitative and qualitative) focusing on program-specific activities and outcomes which work toward context specific improvements.This approach "aims at promoting future performance rather than making judgements on past performance" (Kis, 2005, p. 10).In the strictest sense, these approaches serve different purposes."Since accountability is the main driving force behind quality assurance in higher education, the primary goals of quality assurance processes are to monitor and maintain quality.As a result, quality assurance processes tend to inhibit innovation in teaching and learning rather than advance it" (Nicholson, 2011, p. 8).
The tension between the purposes of, and procedures for, accountability and enhancement in the context of quality assurance in higher education has become problematic on several levels both in research and in practice (Kis, 2005).Areas of concern include: 1) Focus -a singular emphasis on accountability "may damage learning by diverting academic staff's attention away from the improvement of learning, to compliance with the bureaucratic imperative" (Harvey, 1997 as cited by Kis, 2005, p. 13).2) Engagement -a need to incite participation and ownership by faculty members over a process that is meaningful, relevant, and useful to academic programs (Cheng, 2014;Kis, 2005;Kleijnen et al., 2013).3) Decision making -external accountability may impede the ability of institutions to make autonomous decisions about what should be valued and measured in relation to their own missions and identities (Houston & Paewai, 2013).It may be equally ill-suited for external bodies to make decisions regarding program resources and staff development (Kis, 2005).4) Workload -meeting criteria in an assurance process that serves separate purposes, that in some cases have similar needs, may create an unnecessary workload for information likely to be duplicated (Middlehurst & Woodhouse, 1995).5) Reporting -reporting under each purpose is quite different and lack of clarity may lead to misinformation.For instance, programs might hide weaknesses from accountability groups that would be important for the goal of improvement (Kis, 2005).
Numerous scholars in the field of quality assurance are critical about current systems which emphasize accountability (Harvey & Newton, 2004, 2007;Houston & Paewai, 2013;Nicholson, 2011).The common argument is that the design of quality assurance frameworks is often divorced from concerns about the improvement of educational processes.In addition, the design of assurance systems often marginalises the significance of context-specific information that is required for judgements of merit and discussions regarding the advancement of practices (Houston & Paewai, 2013).Taking into account the concerns listed above, perhaps there is a way to bridge both approaches and further integrate a focus on enhancement into existing quality assurance processes.Hodson and Thomas (2003) suggest that individual institutions should have the opportunity to fully engage in the design and implementation of the assurance process to ensure investment in both securing standards and enhancement.This type of investment implies that completing a rigorous enhancement-oriented quality assurance process internally may also serve external bodies monitoring quality standards.Or as stated by Houston and Paewai (2013), "information collated for external accountability might not support internal improvement but information gathered for internal improvement could facilitate external accountability" (p.275).A combination of accountability and enhancement approaches which meets a variety of needs seems to touch on both the fitness for purpose and transformation outlooks on quality.Speaking to the conditions necessary for the latter to succeed, Harvey and Newton (2004) list some "key ingredients" for quality assurance processes (p.161).These include a shift from teaching to learning; the development of graduate attributes; appropriateness of assessment; systems for rewarding transformative teaching and learning facilitation; emphasis on pedagogy; institutional climate to support responsive collegiality; and linkages between quality improvement and learning.Houston and Paewai (2013) add that transformation necessitates faculty participation in "systematic, critical enquiry (or research) in the local context, [which] has the potential to legitimate it as an element of academic work intended to address 'quality' as an academic staff, academic unit, disciplinary and university-level concern" (p.278).Others, such as Cheng (2014), Kis (2005), Kleijnen et al. (2013) and Ramsden (2003), support the notion that faculty participation in quality assurance processes is a powerful factor which contributes to engagement, ownership of the process and change in departmental practices related to revision of student learning outcomes, curriculum design, instructional approaches, and assessment of learning.

Implications of a Participatory Approach to Quality Assurance Defining Participatory Evaluation
In the context of quality assurance and curriculum review, faculty participation is a principal factor associated with successful program enhancement and change (Kis, 2005;Kleijnen et al., 2013;Ramsden, 2003).Stakeholder participation in evaluative processes, or participatory evaluation, is by no means a new area of study.In the field of program evaluation, it has frequently been defined as a collaborative approach to evaluation in which various stakeholders are actively engaged in all phases of the process (Cousins, 1996;Zukoski & Luluquisen, 2002).The participatory approach seeks to share knowledge, develop evaluation skills, and give a voice to all parties and beneficiaries who have a stake in the program.These stakeholders are typically involved in the design of the evaluation, selecting measures, collecting data, interpreting findings, and making and implementing recommendations (Cousins & Earl, 1995;Zukoski & Luluquisen, 2002).
As a form of collaborative social inquiry, participatory evaluation is primarily pragmatic in nature; with a problem-solving and instrumental orientation, its goal is the utility of the knowledge it creates.Additionally, common forms of participatory evaluation often carry political aims to promote fairness among involved individuals that have a stake in the inquiry (Weaver & Cousins, 2004).Cousins and Whitmore (1998) introduced a series of dimensions that are fundamental in characterising different streams of participatory evaluation.Initially, dimensions consisted of control of the evaluation process, depth of participation, and stakeholder selection.In 2007, the latter dimension was replaced with three new dimensions, namely: diversity among stakeholders, power relations among stakeholders, and manageability of evaluation implementation (Weaver & Cousins, 2004).Depending on how a participatory approach is characterised within these dimensions (along a continuum), it may align with one of two distinct streams: Practical Participatory Evaluation (P-PE) or Transformative Participatory Evaluation (T-PE).As the name suggests, the first is concerned with practical problem solving and providing support for organizational decision making and the latter is empowerment oriented and focuses on democratizing social change.Connecting P-PE, for example, to the listed dimensions, Weaver and Cousins (2004) characterise it as control of the evaluation process shared between evaluator and stakeholders, the latter being extensively involved in a variety of tasks and decisions.However, stakeholder diversity is predominantly limited to primary users.Power relations are typically neutral as the interests of the primary users are at the forefront.The limited number of stakeholder groups involved ensures that evaluative processes are easy to manage logistically.
The feasibility of using the process dimensions of a participatory approach in quality assurance is examined in the next part of this section.

Implications for Quality Assurance in Higher Education
Designed principally in the context of evaluations of social programs, the principles and process dimensions of participatory evaluation are equally applicable to the review or academic programs in the context of quality assurance (Martin, Pereyra, Sigh & Stella, 2007;Ondieki & Matonda, 2013;Qin, Fancai & Mei, 2013).According to Cousins (1996): "With its emphasis on collaboration, depth of involvement in all phases of the evaluation and continual interpretation and deliberation of evaluation data," participatory evaluation has strong potential to bring stakeholders together for the purposes of assessing, reflecting, visioning, and enhancing programs (p. 6).Cousins and Whitmore (1998) add that the core premise of P-PE "is that stakeholder participation in evaluation will enhance evaluation relevance, ownership, and thus utilization" (p. 6).
Using the Institutional Quality Assurance Process (IQAP) established by an Ontario University (University of Ottawa, 2011) as reference, it becomes evident that a participatory approach could be used to "critically analyze all aspects of a program, specifically, the curriculum, student population and faculty resources, as well as all other human, financial and material resources [and provide] an in-depth, forward-looking [analysis] based on significant data and on quality indicators" (p.34).An examination of how a participatory approach might be characterised in a quality assurance context using the process dimensions, mentioned above, demonstrates feasibility: 1. Control of the evaluation process can be shared between the administrators of the IQAP and the program managers (departmental chairs) and beneficiaries (such as students, instructors, and public) (Ondieki & Matonda, 2013;Qin, et al., 2013).2. Depth of participation is balanced between the in-depth involvement of a curriculum review committee and the consultative roles of IQAP administrators and program beneficiaries (University of Ottawa, 2011).3. Diversity among stakeholders can be well represented on the curriculum review committee by virtue of having a broad representation of stakeholders such as full and part-time professors, students and support staff, as well as including other beneficiaries in a consultative role (Wolf, 2007).4. Power relations among stakeholders are likely to be quite neutral given that all concerned stakeholders are likely to seek "areas that hold promise for enhancement" (University of Ottawa, 2011, p. 38). 5. Manageability of evaluation implementation may be challenging given that the diversity, and nature, of participation could lead to logistical challenges which may impact process timelines (Weaver & Cousins, 2007).

Potential Contributions of a Participatory Approach
Given the feasibility of a participatory approach to quality assurance processes, characteristics of this approach that would contribute most to an enhancement oriented academic program review would largely be four-fold: 1. Relevance to context -a participatory approach emphasizes that evaluation questions and design are locally relevant and meet the needs of accountability groups, program managers (departmental chairs), and beneficiaries (such as students, instructors, and the public).These stakeholders determine the process of evaluation and implementation of findings (Rabinovitz, 2013;Zukoski & Luluquisen, 2002).2. Engagement of stakeholders -the process sparks creativity and encourages collaborative work among stakeholders, which enables the exchange of ideas and fresh perspectives (Cousins, 1996;Gawler, 2005).

Empowerment
stakeholders are empowered by being fully involved in determining the direction and effectiveness of the evaluation, which encourages stakeholder ownership and dedication to conduct, to interpret and implement an informative evaluation (Cousins & Whitmore, 1998;Rabinovitz, 2013;Zukoski & Luluquisen, 2002).4. Build capacity and sustain enhancementfocus is placed on the construction of knowledge, process, and tools that will allow stakeholders (primarily departmental chairs) to sustain action after the evaluation is completed.This learning equips stakeholders to continue advocacy for change and transformation of their program (Cousins & Whitmore, 1998;Zukoski & Luluquisen, 2002).

Potential Challenges of a Participatory Approach
While a participatory approach to quality assurance can respond to accountability needs and produce findings and learning which can enable sustainable program enhancement, its success hinges on certain conditions.Firstly, the process must be taken seriously.According to Cousins (1996), unless senior administrators of the process consistently support those programs under review (via release time, evaluative expertise, recognition) and promote the participatory nature of the process, the potential of the approach may not be realized.Commitment on behalf of participants is equally key.The involvement of numerous stakeholders (such as instructors, students, support staff, alumni, and employers) in this type of approach takes time, interest, and considerably more planning than traditional compliance-based assurance processes (Cousins & Earl, 1995;Zukoski & Luluquisen, 2002).A significant concern to consider relates to the misuse of evaluation data.With high levels of stakeholder control in the interpretation and reporting of findings, there is a risk that information may be intentionally or unintentionally altered or misused (Cousins, 2004;Cousins & Whitmore, 1998).For instance, a perceived weakness may be omitted in the report to the detriment of program improvement (Kis, 2005).Other concerns include the misuse of power and influence in the selection of participating stakeholders and lack of training and support regarding participatory approaches and evaluation methodology (Cousins, 1996;Cousins & Whitmore, 1998;Zukoski & Luluquisen, 2002).As recognized by Cousins and Whitmore (1998), only through the deliberate inclusion of mechanisms for ongoing observation and reflection of practice will the potential for participatory evaluation be achieved.

Conclusion
This paper has provided an overview of the principal themes associated with quality assurance in higher education: the multiple definitions of quality in higher education, the development and ongoing debate about the meaning and measures of approaches to quality, and the implications of a participatory approach to assurance processes.
In light of the largely accountability-focused quality assurance frameworks, for instance the one used by Ontario Universities, greater stakeholder participation in the various processes involved in higher education quality assurance would enable greater focus on an agenda of program enhancement all while aligning with institutional missions and the needs and expectations of students and government.
Amid the "lack of clarity about what the purpose of quality assurance should be, about the adequateness of diverse methods and instruments used by quality assurance mechanisms, or concerning the consequences of quality monitoring results" (Kis, 2005, p. 33), greater research is needed to explore ways in which assurance processes can stimulate greater commitment toward quality enhancement (Harvey & Williams, 2010).This said, the "application of methods and measures does not, in and of itself, assure quality (Harvey, 2009).Nor does quality assurance, in and of itself, lead to quality improvement."(Houston & Paewai, 2013, p. 277).To make progress, a concerted effort to both situate assurance processes within the context of academic programs and enable a supported participatory approach will greatly contribute to more relevant assurance processes, and by consequence, quality higher education.

Table 1
Accountability and Enhancement Approaches to Quality Assurance in Higher Education *