: A

The COVID-19 pandemic led to an accelerated implementation of digital solutions, such as online proctoring. In this paper we discuss how the use of an ethical matrix may influence the way in which digital solutions are applied. To initiate an ethical discussion, we conducted an online workshop with educators, examiners, controllers, and students to identify risks and opportunities of online proctoring for various stakeholders. We used the Ethical Matrix to structure the meeting. We compared the outcome of the workshop with the outcomes of a proctoring software pilot by examiners. We found that the two approaches led to complementary implementation criteria. The ethical session was less focused on making things work and more on transparency about conditions, processes, and rights. The ethical session also concentrated more on the values of all involved rather than on fraud detection effectiveness.


Introduction
The COVID-19 pandemic has accelerated the already existing trend of increased digitalization in government, commerce, and education.Studies show, for instance, that within education the use of technology enhanced learning has jumped forward in 2020 (van der Spoel et al., 2020).Educational institutions feel pressed to introduce digital solutions such as online proctoring.Despite concerns regarding human values such as privacy, distributive justice, autonomy, well-being, reputation, and reliability, the pressure for fast action may be so strong that it carries with it the risk of unintended negative consequences or backfiring (Stibe & Cugelman, 2016) and ethical blindness (Palazzo et al., 2013).To mitigate this risk, it is important to integrate explicit ethical discussion in the design and implementation process (Van den Hoven, 2017).One tool to support this, is the ethical matrix (Mepham, 2000;van der Stappen & van Steenbergen, 2020).The ethical matrix is a tool that stimulates a closer look at potential risks and opportunities of digital innovations.
At the start of the first corona lockdown in March 2020, Dutch institutes of higher education were pressured to find alternative ways of taking exams.Having entire classes sit an exam in large halls under surveillance of a human proctor was no longer an option.Many courses turned to alternative ways of examination, such as having students write essays.But for some courses, the only viable option turned out to be taking the exam online, with the students sitting the exam from their homes, using their own devices.To prevent fraud during the exam, many institutions turned to online proctoring software.This involves recording the sitting and analysing the recordings afterwards for deviations that might indicate irregular behaviour.The use of this type of software immediately raised questions about privacy and potential unjust accusations.But other human values might be impacted as well.
In this paper we address the following research question: How does the use of the ethical matrix influence the formulation of implementation criteria for proctoring software?To answer this question, we conducted a case study concerning the implementation of online proctoring software to enable online examination.We carried out a pilot test with teachers evaluating the proctoring software in parallel with conducting a workshop with various stakeholders in which we applied the ethical matrix to the case of online proctoring software.From both the pilot and the workshop we collected implementation criteria, which we then compared.
After providing a theoretical background in the next section, we describe the research method we applied in this study in section 3.In section 4 we present and discuss our results, followed by conclusions in section 5.

Theoretical Background
To integrate ethical consideration in the process of design and implementation of digital solutions, we combine the Value Sensitive Design approach (Friedman, Kahn & Borning 2006;Friedman & Hendry, 2019) with the ethical matrix (Mepham, 2000;Mepham et al., 2006;van der Stappen & van Steenbergen, 2020).

Value Sensitive Design
Value Sensitive Design (VSD) originates from the nineties of the last century (Friedman & Hendry, 2019).It is "a theoretically grounded approach to the design of technology that accounts for human values in a principled and comprehensive manner throughout the design process" (Friedman, Kahn & Borning, 2006, p. 349).Human value is defined as "what is important to people in their lives, with a focus on ethics and morality (Friedman & Hendry, 2019, p. 4).In VSD not only the values of the actual users of a technological artefact are considered, but also the values of parties that may indirectly be impacted by the artefact.For example, bystanders, future generations, or individuals who cannot or will not use a service.The values of these stakeholders, as well as potential tensions between these values, are investigated from a conceptual, empirical, and technical perspective.At the conceptual level, the relevant stakeholders and values are identified and defined, based on existing literature and knowledge.At the empirical level, the actual perception of these values by the various types of stakeholders is studied by employing methods such as interviews, focus groups or experiments, leading to further elaboration of the values into norms.At the technical level, the values and norms are translated into technical design or implementation criteria or requirements.The three perspectives are iteratively employed.Over the years VSD has been applied to various domains, including the design of browsers (Friedman, Howe & Felten (2002), wind turbines and wind parks (Oosterlaken, 2015) and AI systems (Umbrello & van de Poel, 2021).DIGITAL SUPPORT FROM CRISIS TO PROGRESSIVE CHANGE In their overview of 20 years of VSD, Friedman & Hendry (2019) discuss 17 instruments and techniques that have been used over the years by various VSD projects.These techniques are either unique to VSD or existing techniques that were adapted to use in VSD.Friedman & Hendry indicate that the list is not exhaustive and that new or newly adapted techniques and instruments are likely to be added over time.They provide, among others, as heuristics for the VSD research and design process to seek an iterative and integrative approach during the entire design process, to use a variety of empirical values-elicitation methods and to continue to elicit stakeholder values throughout the design process as well as apply a value sensitive evaluation process through the deployment phase (Friedman & Hendry, 2019).The ethical matrix as used in this study is an existing instrument adapted to the design and implementation of digital solutions, that can span the entire design, implementation, and deployment process (van der Stappen & van Steenbergen, 2020).

Ethical matrix
The ethical matrix originates from agriculture and was developed to support rational ethical evaluation of biotechnological innovations in agriculture and food production (Mepham, 2000;Mepham et al., 2006).It was developed to support nonethicists in discussing the ethical implications of biotechnical innovations.In the rows of the original matrix the relevant stakeholder groups in biotechnology are distinguished (producers, consumers, treated organisms and biota).The columns distinguish the three fundamental ethical principles of autonomy (deontology), fairness (Rawls) and well-being (utilitarianism).When a biotechnical innovation is under consideration, the ethical matrix is used to discuss the impact of the innovation regarding each of the principles on each of the stakeholders.This impact is captured in the cells of the matrix.Figure 1 presents the original ethical matrix.
The ethical matrix is developed for innovation in the food industry.Since its introduction it has been applied and adapted for various other fields (Vinnari, Vinnari & Kupsala, 2017;Schroeder & Palmer, 2003;(Kaiser, Millar, Thorstensen, & Tomkins, 2007;Kermisch & Depaus, 2018;Chatfield, 2018) In our study we use the adapted version of the ethical matrix as described in van der Stappen & van Steenbergen (2020).In this adaptation the stakeholders are the direct and indirect stakeholders that are identified in the conceptual perspective of VSD.
The ethical principles of the original are replaced by the values as conceptualized in VSD (fig.2).In the cells the potential positive and negative impact of the digital solution on the values of the stakeholders is recorded.

Research Method
To investigate whether the use of the ethical matrix contributes to more value-driven implementation decisions, we conducted a case study at an institution of higher education.The case study concerned the implementation of online proctoring software to enable online examination.The proctoring software concerned makes recordings of the exam sitting of students at home through a webcam and by recording keystrokes.Before starting the exam, the student can be asked to turn her camera around to show the room in which she is taking the exam.During the exam, the software records the students' screens, as well as the students themselves.The images and recordings of the sitting are analysed by an AI algorithm.Any divergent behaviour is reported for further inspection by the examiner.For this to work, the students must install a specific web browser as well as an online proctoring plug-in.
Before implementing the selected online proctoring software, a pilot was conducted with 24 participants (20 teachers, 2 IT professionals, 2 members of the exam committee) who conducted an exam using the proctoring software.The aim of the pilot was to test the usability and effectiveness of the software.Each of the participants answered 12 questions.These included an overall grade for the software, any problems experienced by the participants and the degree of usability and effectiveness the participants attributed to the software.The results of the pilot were translated into implementation criteria which were categorized into requirements, advice, and considerations.
To initiate an ethical discussion about the use of online proctoring and to create awareness about potential undesired consequences, an online workshop was conducted with 10 participants (1 teacher, 1 member of the exam committee, 2 IT professionals, 1 education logistics employee, 1 Digital Learning Environment manager, 2 privacy officers, 2 students).The workshop was led by one of the authors.The aim of the workshop was to identify risks and opportunities of online proctoring for various stakeholders.The ethical matrix was used to structure the discussion.The workshop started with an ethical matrix that already contained the main stakeholders and values.These were identified from literature and earlier discussions with experts (conceptual perspective of VSD).As starting point for the stakeholders, we identified the primary people involved in the processes of preparing, conducting, and evaluating examinations, both on site (as done before the pandemic) and online using the online proctoring software.
The values were selected from lists of values relevant to digitalization (Friedman, Kahn & Borning, 2006;Royakkers et al., 2018), which were compared to views expressed in online posts and publications about the use of online proctoring.The values thus extracted were discussed with IT experts, a teacher, and a student from a different institute of higher education.In the workshop the stakeholders and values identified were validated and the potential impact of the online proctoring software on the identified values for the identified stakeholders was discussed.This was done via identifying potential harms and benefits of using the proctoring software.After the workshop, the results were laid down in a report which was validated by the participants.After validation by the participants, the authors translated the results into implementation criteria.These criteria, too, were categorized into requirements, advice, and considerations.
The implementation criteria of both the pilot and the workshop were combined into one list of 39 criteria.From the list four types of criteria emerged: criteria concerned with facilitation (4), instruction and procedures ( 22), fraud and reputation (10) and logistics (3).
To analyse the contribution of the ethical matrix, we compared the criteria that resulted from the workshop with the criteria that resulted from the pilot.

Results
The average grade given by the pilot participants to the proctoring software was a 7. Problems reported concerned mainly technical problems with installing the required browser or plug-in.Most of the participants concluded that use of the software would be feasible, if necessary, though a few participants doubted its usefulness to detect all fraud.One participant expressed concerns about privacy and other ethical considerations.Based on the pilot 17 implementation criteria were formulated.
In the ethical workshop we started with a matrix containing the stakeholders student, examiner, surveillant, educational institute, programme manager and IT department and the values equality, well-being, reputation, autonomy, privacy, sustainability, and trustworthiness.In the workshop the stakeholder of housemate was added, while the value of sustainability did not generate response from the participants and was subsequently removed from the matrix.Table 1 contains descriptions of the values as well as examples of impact on one type of stakeholder, the student.In a similar manner impacts on the other stakeholders were formulated.Based on the workshop, 25 implementation criteria were formulated to mitigate the potential negative impacts.
We divided the criteria from both sources into three categories: requirements, advice, and considerations.Requirements are criteria that are considered hard prerequisites for implementation.They are not negotiable.Advice includes criteria that are strongly recommended, but not mandatory to proceeding.Considerations are criteria that are considered beneficial but are left to the implementation project to decide on.Examples of each category, one originating from the ethical matrix workshop and one originating from the pilot can be found in Table 2.The reputation of students may be damaged if they are unjustly accused of fraud and records of the accusation are kept.

Ethical matrix
We categorized the criteria into four categories: criteria concerned with facilitation, with instruction and procedures, with fraud and reputation, and with logistics.Table 3 shows the distribution of criteria from the two sources over the categories.A total of 43 implementation criteria were derived from the pilot and workshop together, with an overlap of 4 criteria that emerged from both the pilot and the workshop.Leaving 39 distinct criteria.DIGITAL SUPPORT FROM CRISIS TO PROGRESSIVE CHANGE Analysis of the two sets of criteria reveals a clear difference in focus between the pilot and the workshop.As expected, the criteria from the pilot were more functionoriented, whereas the criteria from the workshop were more value-oriented.
Thus, only the workshop led to criteria regarding the facilitation of students who do not have access to the required hardware or to a suitable space to take the exam (related to the value of equality) and the facilitation of examiners who experience a sudden increase of workload because of the application of online proctoring software (related to the value of well-being).
As for the category of instruction and protocol we found that the criteria from the pilot are focused on providing clear instructions to both students and employees regarding all phases of the examination process, ranging from timely preparation and testing of the technology beforehand to sitting the exam as well as the careful closure of the sitting.The criteria from the ethical workshop are focused on augmenting the protocol with protective measures for students, such as safe online identification, informed consent, right of inspection, dealing with physical disabilities and technical incidents during the exam sitting (related to the values of well-being and privacy).In addition, the workshop led to criteria concerning the long-term effects and feasibility of the online proctoring solution (related to the value of autonomy).
In the category of fraud and reputation, the criteria from the pilot dealt with the fraud analysis effectiveness.The criteria from the workshop dealt with the risk to the reputation of both students (incorrect signalling by the algorithm of potential fraud) and institute (mistakes in the process, reduced perceived value of exam result, privacy breach).
Finally, in the category of logistics, the pilot led to criteria concerning the suitability of online proctoring software for various types of exam, whereas the workshop focused on the feasibility of the entire process of online proctoring (value of wellbeing).

Conclusion
In this study we investigated whether the use of the ethical matrix as adapted by Van der Stappen & van Steenbergen (2020) enriched the outcomes of a functional pilot concerning the formulation of implementation criteria of online proctoring software.We expected that the explicit focus of the matrix on the values of various stakeholders would generate additional criteria.The analysis of the two lists of criteria generated from the pilot on the one hand and the workshop using the ethical matrix on the other hand, confirmed that the two approaches lead to different types of criteria.
We conclude that the pilot and the ethical session are complementary.The pilot led to implications focused on function, whereas the ethical session provided insight into value-oriented requirements.We believe that in educational institutes value and function are equally important.By allocating a workshop to formulating ethical requirements and considerations early in the process, the importance of both function and values can be considered during the implementation.The ethical matrix appears to be a very useful instrument in facilitating and structuring discussions on values by non-ethicists such as educators and students.
Our study concerns only one case which of course limits its potential for generalization.We believe, however, that the results are promising.Increased application of the ethical matrix in a diversity of contexts will hopefully lead to more comparative analyses in the vein of our study.Besides providing increasing insight in the effects of applying the ethical matrix, we are hopeful that it will also contribute to implementations that are more sensitive to the values of all stakeholders concerned.We intend to study how the ethical matrix can also be used to test this, by applying it again after having conducted online proctoring for some time, as proposed in Van der Stappen & Van Steenbergen (2020).
We believe that the use of the ethical matrix might add the dimension of impact to the widely accepted dimensions of functional and non-functional requirements in digital application.