e -Proctoring exams at a regional university during COVID-19: Understanding the challenges

The ongoing Coronavirus (COVID-19) pandemic has had an effect on most sectors in the countries affected by Covid-19 cases. Even in the South Pacific region, countries were forced to go under lockdown, flights were cancelled, businesses, schools and universities closed. Similar to most institutes in the region that were forced to close, the University of the South Pacific (USP) cancelled its face-to-face (F2F) studies and instead switched to Emergency Remote Teaching (ERT) at all its 14 regional campuses. It is undeniable that teaching using ERT has challenges. However, teaching using ERT at a one-campus university may be easier than teaching at a regional university since it brings with it more challenges due to the geographical distance and the gap in support available at these campuses. USP faced similar challenges since academics could not travel to regional campuses for tutorial support. All support, tutorials and lectures were conducted online using BigBlueButton (BBB) or Zoom. That, in itself led to additional online training of staff and students to better familiarize them with online learning and teaching. Courses offered at USP have used Moodle as its learning management system (LMS) since 2006. ERT during COVID-19 lockdowns meant courses that were offered F2F were switched to online. Switching from F2F, courses to Online mode during lockdowns in 2022 and 2021 did not hinder much with the staff and student preparedness since all courses were on the LMS. The only major change that needed to be introduced was the inclusion of online assessments in these courses. One of the components for online assessment included the introduction of e-proctoring as a system to supervise students attempting online exams to meet the requirements of the accrediting bodies. There are various e-proctoring software that can be used for such supervisions. USP used Proctorio during the recent lockdowns. This paper shares the experience of testing, reviewing and getting students ready for online assessments using e-proctoring, as well as adjusting the LMS to support the online assessment navigation system. The paper additionally discusses the success and challenges faced while using e-proctoring for such online assessments.


Introduction
COVID-19 cases began to be reported in Wuhan, Hubei Province, China at the end of 2019 and by late March 2020, cases began to be reported in the South Pacific region. The pandemic had major impact on all countries affecting the business, tourism, and education sectors the most. Due to a surge in COVID cases education sector in most countries had either to suspend or transform their educational programs. Where the education sector is concerned, governments around the world had to make the difficult choice between students' and teachers' health and face-to-face learning. The decision to move to online teaching was not an easy one for many educational institutes given the speed with which the decision to change the mode of study had to be made. Hodge, Moore, Lockee, Trust and Bond (2020) appropriately coined the term ERT due to the nature and speed with which the shift of instructional delivery occurred during the pandemic. Unlike a well-planned and thought through online teaching, ERT could be termed as a strategy used to survive during a crisis. COVID-19 was a crisis, unlike the numerous other natural disasters that the whole world, including the USP region had experienced in the past.
USP is a regional tertiary educational institute with campuses in all the 12 countries that are its members. At any given time during the year, one or the other form of disaster can be experienced by any of these countries, whether it is flooding, landslides, hurricanes, cyclones, earthquakes, tsunami, political coups or volcanic eruption. However, very rarely have so many campuses been affected simultaneously and where all other sectors were brought to a standstill as well. USPs experience with multi-modal teaching gave it the confidence to move quickly to ERT during the COVID-19 pandemic. USP offers courses and programs beginning from Preliminary to Post Graduate using any or all of the modes that it offers; F2F, Print, Online, Flexi learning, and Blended mode with ability to switch between modes of study that suits its campuses, staff and students. This unique ability to switch between modes of study came very handy during COVID-19 pandemic. This paper discusses the strategies USP used to continue teaching without any major cancellations of courses or programs that it offers, and the alternative measures it took to overcome the lack of F2F assessment practices during the COVID-19 pandemic.

Literature review
World Health Organisation (WHO) declared Coronavirus in March 2020. Since it was highly contagious and with high mortality rate, countries all over the world imposed strict protocols as a precautionary measure to reduce high human-to-human infections. Some of the measures taken by countries included complete or partial lockdowns, social distancing, and curfews. Because of such harsh regulations more than 90% of enrolled students all over the world experienced disruptions to their studies (UNESCO, 2020a;2020b;UNICEF, 2020).
In order to control the spread of the virus countries in the South Pacific region began to isolate their countries. Strategies such as complete border shutdowns, and strict travel restrictions were imposed (Diarra, Muna and Diarra, 2021). By June 2020, only one out of the 12 USP countries had any Covid-19 cases. The case unfortunately was not the same in 2021 when a more rapid spread of the Delta variant was seen in the region and a harsher, longer-term lockdown and curfew was seen.
The 2020 Covid-19 lockdown and restrictions had given the South Pacific region a bit of time to plan for the next wave. USP had a short switch to online teaching for the second half of semester one. With no cases of Covid-19 reported in the region, a back to normalcy was seen in the second half of 2020 when F2F teaching was reintroduced.
COVID-19 had pushed many educational institutions across the globe to adopt course online learning and have e-proctored tests and exams (Hussein et al., 2020;Mishra et al. 2020). This shift allowed the students to complete their formative and summative assessments in order to complete the requirements of the course. e-Proctoring is not a new area of research (Nigam et al., 2021). Universities around the world started implementing e-Proctoring systems for online exams prior to COVID-19 and the harsh lockdowns that took place. A proactive approach to implement e-Proctoring helped USP in ensuring students were able sit for supervised tests and exams even when there were lockdowns primarily for the accredited courses and programmes. e-Proctoring involves using computer-generated tools to monitoring student activities during assessment activity. This allows students to take their tests from any location of their choice. The e-Proctoring system helps ensure integrity and reliability by authenticating students during the exam (Foster & Layman, 2013). A study by Dendir & Maxwell (2020) found that there was a decrease in average performance in courses that engaged the e-Proctoring system. This could be interpreted as an indication that students were cheating in online tests prior to the implementation of the e-Proctoring system. However, some authors found no significant difference between the proctored test scores compared to non-proctored online tests (Ladyshewsky, 2015;Beck, 2014). This paper presents first-hand user experience of setting-up and taking an e-proctored test using Proctorio.

Region and USP
USP is a regional university that was established in 1969 when the newly independent countries began to realise the need for skilled and professional citizens to govern their countries. Twelve countries (Fiji, Tonga, Samoa, Cook Islands, Kiribati, Nauru, Niue, Tokelau, Tuvalu, Vanuatu, Solomon Islands and more recently Marshall Islands) collectively own the university. A university Council governs the institute. There are campuses and subcampuses in all these countries. Three large campuses (one in Suva, Fiji; another in Apia, Samoa and in Port Vila, Vanuatu) have provide accommodation for students as well. They also teach many courses F2F at these campuses, while smaller campuses offer courses by Print mode. Occasionally flexi schools are conducted at smaller campuses giving the smaller island nation students get a feel of F2F learning.
Laucala campus, based in Fiji is the main campus of USP. All administration, large science laboratories, computer labs, a fully functional library can be found here together with most of USP's teaching staff. The member countries totally depend on the teaching and learning support from Laucala campus for all courses, regardless of the mode of study.

Modes of study at USP
When USP was established in 1969, it began by teaching courses F2F. By 1972, Print mode was introduced to offer courses for those students who could not afford to travel to the Laucala campus to study. Later, in 2006 online mode was introduced with the use of Moodle as the Learning Management System (LMS). Therefore, not only do all teaching staff at USP have some experience of teaching by more than one mode, but they are also familiar with USP region's complexities and the facilities available at all the campuses.
Currently, all courses, regardless of the mode of study, use Moodle for course management. That experience of using a LMS became useful for academics at USP during the first lockdown in 2020 as well as during the longer lockdown in 2021.

COVID-19 situation and Emergency Response Teaching (ERT) e-Proctoring at USP
USP had a significant growth in online and remote learning due to COVID-19. The increase has led the academics to use various approaches in assessing these learners. This shift brought about newer challenges that included student authentication and reduction of potential cheating in online exams. e-Proctoring of online assessments could be one of the few solutions available.
A 'proctor' is a supervisor of students during an examination session. With online e-Proctoring, the supervisor need not be a human being onsite. The e-Proctoring system that is being used at USP is Proctorio. It provides a fully automated e-proctoring option that does not require human supervisors. Proctorio allows the students to take their online exam from remote locations, while ensuring the integrity and reliability of the exam by authenticating the students. (https://Proctorio.com).
The Centre for Flexible Learning (CFL) at USP conducted tests on how this service could be used during online examination keeping in mind the connectivity issues in the USP region.

1.
A mock online test was created for the AF432 and AF439 (Post Graduation 400 level Accounting courses) students in their respective courses after the Moodle integration was done by ITS. 2. All the Proctorio settings were thoroughly checked and tested prior to the mock and actual test/final examination. 3. A comprehensive student guide was created and given to the students prior to the examination. 4. A survey was also created to collect the student experience of using the Proctorio system (during mock and actual tests). 5. Live student support on Zoom was provided to the students. The zoom link was provided to the students which they could use if they had issues with the proctored exam. 6. A debrief was organised (after the mock trials) with the course lecturer to discuss the issues encountered by the students and how to overcome this in the final examination.

How does Proctorio work?
Proctorio only works with Google Chrome web browser. Students are required to install the Proctorio's chrome extension. Proctorio is only active when a student takes an e-proctored test, and does not interfere with other Moodle tools or plugins. Proctorio records the movements and sends the video and other data to the teaching staff for review. Proctorio will flag all suspicious activities that are not allowed during the exam. Course lecturers will be able to review the video and data.

Preliminary Tests
The live trials began after the team conducted successful in-house testing with staff members. This gave the team a chance to experience using the system from a student's perspective. All the issues were noted and a checklist on 'taking a proctored test' was prepared and given out to students (checklist included: technical requirements, examination requirements, support during examination and some tips on taking a proctored exam).

Training of trainers (Lecturers)
The proctoring team organised a training session for the two academic staff of AF432 & AF439. This gave them the opportunity to have first-hand experience of the proctoring system. This also gave the team a chance to experience using the system from a student and lecturer's perspective. In addition, it helped them understand what to expect on the actual examination.
The Proctorio behavioural settings were also discussed with them to give them a clear picture of the systems possibilities.

Live Mock Tests
Setting up the mock trials before the short test was a way to introduce the tool to the students. This helped iron out some of the issues that the students faced before the actual examination. The issues encountered by the students were recorded and the proctoring team discussed solutions with the lecturers. Table 3 below shows the number of students who were able to successfully attempt the mock test. The students used the student guides that were provided to them in the course. The students successfully installed the Proctorio plugin and were able to take the mock test. After the tests concluded, the team asked the participants to fill in a short evaluation questionnaire that would help redesign the proctored tests in the future. 149 (out of 254 test takers) results were collected and analysed. The results are as follows:

Figure 1: Overall experience of students taking e-proctored exam
It can be noted that approximately 90% of the students liked the overall experience of the e-proctored exam and about 10% felt otherwise.

Figure 2: Best thing about having e-proctored exam
Approximately 72% of the students liked the idea of taking the exam from home. In addition, a further 13% of the students liked that their exam was easily accessible.

Figure 3: Ideal environment for e-proctored exam
Approximately 72% of the students liked the idea of taking the exam from a choice of their own. However, 28% of the students indicated that having exams at a testing centre is better.

Figure 4: Would you take another e-proctored exam?
Approximately 84% of the students liked the idea of having an e-proctored exam. However, approximately 21% of the students did not like the idea of e-proctored exams.

Live Trial in AF432 Test 2 (10% of student's coursework)
Having a proctored Test 2 (assessed) in AF432 gave the students an experience and confidence for their proctored final examination. The Test 2 had a slightly high uptake of the proctored test, as it was contributing towards their final grades. Table 4 below shows the number of students who were able to attempt the actual Test 2 in AF432. The students were provided with live support and they used the student guides that were provided to them in the course. The students were able to install the Proctorio plugin and were able to take the mock test. After the actual AF432 Test 2 was concluded, the team asked the participants to fill in a short evaluation questionnaire that would help redesign the proctored tests in the future. 95 (out of 153 test takers) results were collected and analysed. The results are as follows:

Figure 5: Overall experience of students taking e-proctored exam
It can be noted that approximately 85% of the students liked the overall experience of the e-proctored exam and about 15% felt otherwise.

Figure 6: Best thing about having e-proctored exam
Approximately 87% of the students liked the idea of taking the exam from home or liked the idea that their exam was easily accessible.

Figure 7: Ideal environment for e-proctored exam
Approximately 80% of the students liked the idea of taking the exam from a choice of their own. However, 20% of the students indicated that having exams at a testing centre is better.

Figure 8: Would you take another e-proctored exam?
Approximately 78% of the students liked the idea of having an e-proctored exam. However, approximately 21% of the students did not like the idea of e-proctored exams.

Final Examination (50% of student's course total)
Having taken at least one practice proctored test gave the students some experience and confidence for their proctored final examination. Table 5 below shows the number of students who were able to attempt the actual Final Examination in AF432 & AF439. The students were provided with live support and they used the student guides that were provided to them in the course. The students were able to install the Proctorio plugin and were able to take the mock test. After the actual Final Examination was concluded in AF432 and AF439 the team asked the participants to fill in a short evaluation questionnaire that would help redesign the proctored exams in the future. 114 (out of 273 test takers) results were collected and analysed. The results are as follows:

Figure 9: Overall experience of students taking e-Proctored exam
It can be noted that approximately 84% of the students liked the overall experience of the e-proctored exam and about 16% felt otherwise.

Figure 10: Best thing about having e-proctored exam
Approximately 89% of the students liked the idea of taking the exam from home or liked the idea that their exam was easily accessible.

Figure 11: Ideal environment for e-proctored exam
Approximately 67% of the students liked the idea of taking the exam from a choice of their own. However, 33% of the students indicated that having exams at a testing centre is better.

Figure 12: Would you take another e-proctored exam?
Approximately 86% of the students liked the idea of having an e-proctored exam. However, approximately 14% of the students did not like the idea of e-proctored exams.

Laptop Issues
One of the issues that was highlighted by the students (who were mostly from banks and other financial sectors) was that they were using their office laptops that prevented them from installing any plugins or cameras. To mitigate this issue, the students were issued with laptops by the Information Technology Department at USP to be used for their mock and actual e-proctored examination.

Internet Issues
Most businesses were operating from home in Fiji (Lami to Nausori) during the time when these e-proctored exams took place. Having a reliable internet connection in these areas was a challenge, as connections were basically stretched to their limits.
Since this was an on-going issue, the team met with the course coordinators and convinced them to open the tests for a longer period. This allowed the students to take the tests in the evening when the connections were a little better.

Other issues
1. Proctorio requires the use of the Google Chrome web browser, some students had to install the required browser.
2. A Proctorio Chrome extension was required. It took time to install and in one instance, it took almost 20 minutes to have the extension installed.
3. Proctorio enables passwords in the quiz and this created a lot of confusion amongst the students. They were not required to enter a password if they went through the pre-checks before the exam.
4. Students who dropped off quiz because of power failure or internet issues were required to go through the pre-check again (this meant losing extra minutes for their exam).

5.
Taking the picture of the ID card was difficult for some of the students.
6. Dark mode on USP's Moodle created readability issues (blue on black became very difficult to read). Therefore, the students were advised to use light mode.
7. Proctorio pre-checks required the students to do the following before they could proceed to the exam: a. close all other Browsers and Tabs that were open b. power supply to be plugged in (even when the laptop was 96% charged) c. disable the secondary monitor d. share screen with proctor e. allow access to microphone and webcam 8. 'CMID error' appeared for some of the test takers, but this was later rectified.
9. Stopping the 'sharing the screen' option with proctor will terminate the test.

Conclusion
Overall, the students' feedback on the use of the Proctorio system was positive. The results also show that the proctoring system presents a potential solution to the issue of student authentication and cheating in online examinations. In addition, there were concerns about the student data protection and the impact that feeling 'watched' might have on their online assessment experience.
Since this was the first time a proctoring system was tested out in live courses, the proctoring team provided live student support on Zoom. However while the students were taking the examination there were no invigilators. The test authentication and invigilation process took place via the Proctorio system. Having a completion rate of 96.13% is a great indication that the tool was well received by the students.
The security of online examinations does not relate so much to keeping out intruders, but rather to authenticating the students (Lilley et al., 2016) to ensure that the test is completed by the registered students themselves and not someone else.
The team plans to carry out a further study that will have larger groups of students to gain a deeper understanding of our students' attitudes/perception towards the proctoring system.
In addition, creation of an online test procedure would help the lecturers in analysing the incident report that Proctorio will be providing after the students have taken the test.
Finally, the use of the proctoring system would allow for greater flexibility in terms of the assessment formats used in remote, online or distance courses at USP. It would make it possible for online or distance students to take part in timed supervised tests.

Recommendations
1. Use light mode on Moodle when taking proctored tests.

2.
A demo/trial/practise test for students is highly recommended so that they get familiar with the requirements of e-proctored tests.
3. Students should be informed of Proctorio's live chat feature, as it was very helpful during the testing phase.
4. Request that Proctorio remove the password instructions on the quiz page as it creates anxiety before the test.
5. USP should develop a University-wide required pre-defined behavioural settings for e-proctoring that could be used during online tests. This would require discussions with Schools, Office of DVC-Education, Student Administrative Services (SAS), and Information Technology Services (ITS).