Asynchronous Delivery of a 400 Level, Partially Peer-Graded, Oral Presentation and Discussion Course in Systems Neuroscience for 60 Students during the COVID-19 Pandemic

Abstract

A 400-level undergraduate oral presentation and discussion course in Systems Neuroscience was delivered asynchronously online during the COVID-19 pandemic. Enrolled students banked their narrated oral presentations in video format online then engaged in peer evaluation in virtual classrooms through the course website. Student delivered their oral presentation and responded to peer questions at their leisure and convenience, without the stress and anxiety associated with a “live” performance delivery in front of their peers. A remote and asynchronously delivered course facilitated much more peer contact than “live” versions of the course, which included a total of 62 uploaded presentations, 301 video responses uploaded to 1985 questions posed by peers, a total of 1159 feedback questionnaires submitted, 1066 rankings submitted of viewed oral presentations, and 1091 scores submitted evaluating the quality of questions posed by reviewers of oral presentations. A major drawback in the remote, asynchronous deliver was the enormity of peer engagement through the course website portal, which was mostly blind to the instructor because of the inability to effectively cross-index data linked between the student entries in the LEARN course website and the uploaded videos stored within BONGO Video Assignment tool. Nonetheless, a consistent engagement of students, and the positive feedback from enrolled students, indicate that a future version of this oral/written discussion course will be delivered, in part, remotely and asynchronously, even without a mandated delivery of the course by a remote and asynchronous method due to the COVID-19 pandemic restrictions in 2020–2021.

Highlights

• isolation and lack of learning opportunities with peers during the COVID-19 pandemic
• course centered on peer communication in remote, asynchronous learning environment
• online “How-To” videos guided students in 400 level undergraduate course
• communication between oral presenter and 4 questioners in 244 virtual classrooms
• self-grading from 1066 peer rankings of 363 video uploads, 1985 posted questions

1. Introduction

By Spring 2020, it was evident that an in-person, 400 level undergraduate, communication course in Systems Neuroscience in Fall 2020 would have to be adapted for remote, asynchronous delivery in an online environment to enrolled students because of the COVID-19 pandemic. Challenges of the COVID-19 pandemic in 2020 and 2021 to teaching [1,2,3], including the undergraduate medical [4,5,6,7] and neuroscience curricula [8,9], have been documented by many. Online learning experiences are often solitary learning experiences consisting of following textbook reading assignments, and downloading and watching lecture videos. Student learning is usually assessed through completion of online quizzes/exams.

Forced social isolation and lack of a daily structure of in-person attendance in-person classes during the COVID-19 pandemic, deprive students of regular peer contact and learning opportunities from their peers.

What we present here is how we created an online course which was structured to facilitate regular peer engagement and evaluation, in spite of the remote and asynchronous learning environment due to the COVID-19 pandemic, and working with a ballooned enrollment of ~sixty students.

Here we discuss the granular details of how a senior, 400-level oral presentation and discussion course was ported to an online environment, with delivery of oral presentations and response to questions amongst 60 enrolled students, using LEARN (D2L) course website portal and BONGO Video Assignment tool. We were able to deliver an accessible, mostly self-guided, asynchronous oral presentation and discussion course, amongst 60 enrolled students. We discuss both positive and negative issues related to working with the BONGO Video Assignment tool. A major issue we encountered is the lack of integration, and ability to cross-search, and easily collect data between uploaded/entered information by students using the BONGO Video Assignment tool and what was entered by students using available tools within the LEARN (D2L) course website. Macro scripts were programmed to automatically log in and harvest data from the LEARN (D2L) course website so each student could receive a weekly update of ongoing peer evaluation results by quizzes entered by oral presenters and questioners of oral presenters.

Overall, student engagement during the course was high, based on the consistent student participation in fulfilling required tasks during the course. Voluntary student feedback on the course was limited, but overall suggested that the course was a positive, unique and important learning opportunity for them.

While it does not replace the learning gained in “live” face-to-face classroom discussions, it is possible to facilitate meaningful and regular peer engagement and learning within a remote, flexible, asynchronous learning environment, to potentially hundreds of enrolled students, run by only a single lecturer and a couple of teaching assistants.

2. Material and Method

2.1. Online Remote Learning Tools

A course website portal was created within LEARN (Desire2Learn, Kitchener, ON, Canada), containing embedded Video Assignment tool, BONGO (BONGO, Loveland, CO, Canada). Weekly synchronous student help sessions were communicated by video conferencing using Microsoft Teams.

2.2. Course Schedule

PART A: Each enrolled student chooses one unique research topic out of 180 available research topics within the first week (Appendix A—Listing of available research topics and instructions in how to choose a research topic)

PART B: Enrolled students then have five more weeks to research their chosen research topic (Appendix B—Instructions in how to research your research topic (over weeks and weeks of preparation) and then:

• post a referenced “Nature-style” scientific abstract (must be < 750 words) as a “New Discussion Thread” within LEARN course website and append associated PDF documents of 1 or 2 review articles, and 3 to 5 scientific research articles containing experiments, pertaining to their abstract and oral presentation to their Discussion Thread (Appendix C—Instructions in how to write your research abstract; Appendix D—Instructions in how to post your research abstract).
• upload a MP4-formatted narrated PowerPoint oral presentation on their research topic within BONGO Video Assignment Tool (Appendix E—Instructions in how to create a narrated oral presentation video in MP4 format).

PART C: Each student evaluates and responds to questions of four oral presentations they view per week, carrying out tasks in the following order (2.3.1 to 2.3.3, below), repeated over five weeks.

• Each student has one week to view four oral presentations within their Video Assignment Folder (provided randomly from the pool of all oral presentations in BONGO) and to submit at least one thoughtful and unique question as “Reply” to “Thread” under the research abstract entry of each of the four oral presentations assigned to each student. (Appendix F—Instructions in how to prepare and ask at least one thoughtful and unique question in a “Reply” to “Discussion Thread”).
• Within three days henceforth, the oral presenter is required to post a video response addressing the questions posed by the four peers who viewed the oral presentation and posted questions under the “Reply” to “Thread” (Appendix G—Instructions in how to find your questions posted of your oral presentation and how to create and upload a question response video).
• Within three days henceforth, both oral presenters and questioners submit evaluations of each other:
• Evaluation of questioners: Oral presenters grade each questioners of their presentation that week on whether they had been asked at least one thoughtful and unique question in response to viewing their oral presentation. (Appendix H—Instructions in how to measure and evaluate the quality of questions of your oral presentation from questioners).
• Evaluation of oral presenters: Each of the four reviewers/questioners of oral presenters rate the quality of the oral presentation materials, the quality of oral presentation delivery, and the ability of the oral presenter to respond to questions (on a 1 to 5 scale). There is also mandatory entry of constructive feedback submitted to the oral presenters which they viewed that week (Appendix I—Instructions in how to evaluate oral presenters by feedback questionnaire rubric). In addition, all reviewers/questioners rank the four oral presentation which they viewed that week in order of: Best, Second Best, Third Best, and Least Favorite. (Appendix J—Instructions in how to rank the four oral presentations that you have watched and assessed during the week).

PART D: During the final exam period, students submit a final research report based on their submitted abstract and oral presentation topic (Appendix K—Instructions in how to draft your final written submissions (serving in lieu of a final exam) and how they will be assessed) and submit a standard research essay on a unique systems neuroscience topic not chosen as oral presentation topics by students, assigned to them within the first six weeks of the course (out of the pool of 180 available research topics, Appendix A—Listing of available research topics and instructions in how to choose a research topic).

2.3. Student Guidance

• A set of 10 upload How-to videos to the LEARN (D2L) course website provided students with visual instructions on how to carry out the required tasks during the course:

Video #1: How to choose a research topic (1 min, 29 s)

Video #2: How to research your research topic (over weeks and weeks of preparation) (25 min, 28 s)

Video #3: How to write and post your abstract on your research topic (25 min, 43 s)

Video #4: Instructions for writing your written submission and how they will be assessed (14 min, 39 s)

Video #5: How to generate, finalize and upload your oral presentation (8 min, 52s)

Video #6: How to prepare and ask a question or two after watching an oral presentation (3 min, 1 s)

Video #7: How to find your questions of your oral presentation and how to create and upload a question response video (8 min, 15 s)

Video #8: How to measure and evaluate the quality of questions of your oral presentation from questioners (3 min, 47 s)

Video #9: How to fill feedback questionnaire rubric of oral presentations (2 min, 38 s)

Video #10: How to rank the four oral presentations that you have watched and assessed during the week. (1 min, 29 s)

• Sample written abstracts, research reports, and videos on how to create oral presentations and video responses to peer questions were created by the teaching assistants or previous enrolled students in the course (with their copyright permission) were available for downloading and viewing by enrolled students.
• Regular, usually daily mass emails were sent out to students during the course providing guidance and reminders of impeding deadlines.

2.4. Peer Evaluation Entries Were Automatically Mined from the LEARN (D2L) Course Website, and Sent as Email Reports to Each Student Using Python Scripting Language

Over a five-week period in the latter half (Part B) of the course, each student posted questions of the four oral presentations that they viewed over the week under the appropriate research topic Discussion Thread, and then each oral presenter uploaded video responses to the questions posed by four different students under their research topic Discussion Thread. By the end of the week, the oral presenters and the questioners of the oral presenters were to evaluate each other by online quiz questionnaire.

We ensured that all 60 students received individual emails every week as a student’s notebook of their collated peer evaluation feedback by coding in Python, a scripting language, with instructions to automatically log in and collect reports generated in the LEARN (D2L) course website (requestReports.py) and then download these reports of student data entries contained in the LEARN (D2L) course website (downloadReports.py). Different scripts in Python were coded to automatically assemble the peer evaluation reports downloaded from the LEARN (D2L) course website into spreadsheets to generate a weekly feedback report (generateWeeklyReports.ipynb), updated as a student’s notebook of aggregate peer reviews over the many weeks of peer evaluation (MarkingNotebook.ipynb). Then this peer review feedback report was automatically appended to individual emails and sent to each student (send Emails.py). We provided diagrammatic representations (flowcharts) of the Python script sequences and associated folder structures used to data mine the Learn (D2L) course website for student peer review entries and collated the data entries into individual student notebooks for sending to each student by email (Appendix L—Flowcharts of Python programming script sequences and folder structure for automatically sending weekly peer review feedback reports to enrolled students). In the Supplemental Material, we have appended our Python script files, including a requirements list, a Pipfile, 3x Python files (.py), and 2x Python notebooks (.pynb).

2.5. Remote Learning Help for Students

One of the challenges in remote learning is a dependence on a students’ individual internet connectivity to the course website portal (from anywhere in the world). This course depended on success in uploading and downloading hundreds of videos up to 30 min in length, processed through the BONGO video assignment tool. Individual student issues in remote learning help were forwarded to the Computing Helpdesk.

3. Results and Discussion

3.1. Analyses of Student Retention in a 400-Level Asynchronous, Remote Learning Oral/Written Communication and Discussion Course

Many standard courses, such as in-person laboratory courses, had been canceled for Fall 2020, ballooning pre-enrollment to 203 students for this 400-level course normally capped at an enrollment of 25 students. The final enrollment was 60 students, with 143 students de-enrolling mostly within the first few weeks of the course. Of the 203 students who had pre-enrolled and/or shadowed the course for up to many weeks, only 29.6% (or 60 out of 203) of the original number of interested, pre-enrolled students remained and completed the course and received a final grade. It was not surprising that a large fraction (70.4%) of students who sampled and shadowed the course briefly would not be agreeable to completion of the course upon discovery upon admittance that it is an intensive oral presentation and discussion course with three sets of required writing submissions in fulfillment of the course requirements.

By week six, there were a total of 62 enrolled students who submitted a written abstract and uploaded their corresponding oral presentation video on a unique, chosen research topic to the LEARN (D2L) course website. There was only a 3.2% (or two out of 62) attrition rate of enrolled students dropping out of the course after submission of their research abstract and uploading of their oral presentation video by week six. A high 96.8% retention (60 out of the 62 who completed Part A) fulfilled the second half (Part B) of the course.

3.2. Analyses of Student Engagement in 400-Level Oral/Written Communication and Discussion Course Delivered Asynchronous by Remote Learning

Overall, we measured a consistent engagement in students completing required tasks throughout the 12-week course (see

Table 1. Assessment of student engagement in 400 level oral presentation and discussion course carried out asynchronously by remote learning.

	Part A	PART B (Peer Review Period of Oral Presentations)
	Week #6	Week #7	Week #8	Week #9	Week #10	Week #11	Total
Task Carried out by Students	Oral Presentation Submissions	Video Responses Uploaded to Peer Review Questions of Oral Presentations
# of video uploads/expected # of video uploads (%)	62/62 (100%)	61/61 (100%)	61/61 (100%)	61/61 (100%)	60/60 (100%)	58/60 (96.7%)	363/365 (99.5%)
task carried out by students		Questions posed by peer reviewer of oral presentations
average number of questions posed by peer review of oral presentations (expectations of minimum of one)	N/A	1.57	1.68	1.67	1.67	1.60	1.64
total number of questions posed by peer review of oral presentations (expectations of minimum of one)	N/A	383	410	407	400	385	1985
task carried out by students		Peer review feedback questionnaires completed for oral presentations
average/expected average of feedback questionnaires received for each oral presentation	N/A	3.98/4.00 (99.6%)	3.94/4.00 (98.4%)	3.68/4.00 (91.9%)	3.82/4.00 (95.6%)	3.71/4.00 (92.7%)	3.83/4.00 (95.6%)
total number of feedback questionnaires received for each oral presentation	N/A	243	240	224	229	223	1159
task carried out by students		Peer review student rankings completed for oral presentations
average/expected average of student rankings received for each oral presentation	N/A	3.69/4.00 (92.3%)	3.61/4.00 (90.3%)	3.65/4.00 (91.1%)	3.55/4.00 (88.7%)	3.08/4.00 (77.0%)	3.52/4.00 (87.9%)
total number of student rankings received for each oral presentation	N/A	225	220	222	213	185	1066
task carried out by students		Peer review student evaluations of whether oral presenters believed that questioners posed a thoughtful, unique question of oral presentations that they reviewed
average/expected average of peer review evaluations of questioners of oral presenters	N/A	3.76/4.00 (94.0%)	3.79/4.00 (94.8%)	3.73/4.00 (93.1%)	3.52/4.00 (87.9%)	3.21/4.00 (80.2%)	3.60/4.00 (94.8%)
total number of peer review evaluations of questioners of oral presenters	N/A	229	231	227	211	193	1091
total number of tasks carried out by students	62	1142	1162	1142	1113	1043	5664

). In Part A of the course, enrolled students by the end of the first week of the course had all chosen a unique Systems Neuroscience research topic within the LEARN (D2L) course website out of possible research topics contained within a “Group”, containing a maximal enrollment of one student per each of 180 possible research topics.

A total of 62 oral presentations were uploaded of an average length of 23.8 min out of the maximum allowable 30 min of uploaded video length after the end of Part A by Week #6. In the second half of the course, Part B, students were to post questions to the oral presenter after the peer review of oral presentations.

While we requested a minimum of one question posed for each peer reviewed oral presentation video under the appropriate Discussion Thread for each research topic, students posed an average of 1.64 (range: 1.57 to 1.68) questions per week over the five-week oral presentation review period. Overall, there was a staggering 1985 peer review questions posted, ranging from 223 to 243 questions posed per week over the five-week oral presentation review period by the 60 or 61 students enrolled in the class.

We requested that reviewers of oral presentations fill-in peer review quiz questionnaires providing personal feedback to oral presenters and also carry out a student ranking of the four oral presentations that each student viewed per week over the five weeks. Engagement in the oral presentation review process was consistently high over the five-week oral presentation review period, with a completion rate of fill-in peer review quiz questionnaires of 95.6% (range of 92.7% to 99.6%), and a completion rate of student rankings of oral presentations reviewed per week of 87.9% (range of 77.0% to 92.3%).

There was also a consistently high rate of oral presenters filling in quiz evaluations answering whether questioners of their oral presentation had asked at least one thoughtful and unique question. This completion rate of evaluation of questioners of oral presentations was 94.8% on average, ranging from 80.2% to 94.8% per week.

3.3. Positive Aspects in Using the BONGO Video Assignment Tool within the LEARN Course Website for Delivery of a 400-Level Asynchronous, Remote Learning Oral/Written Communication and Discussion Course

We had to have a means for the 62 enrolled students to receive a randomized sample of 20 out of the 62 videos to peer review over five weeks in Part B of the course. Using the BONGO Video Assignment Tool, under Submit > Video Assignment > Configure >Show Advanced, individual Video Assignments could be configured to “Automatic (System Selected)” for Peer Review. This meant that the least viewed of the pool of 62 oral presentation videos at any point in time would automatically be sent for peer review in a student’s “empty” video assignment folder, that is a student’s video assignment folder lacking a video which the student had not peer reviewed yet. Once students pressed to view an oral presentation and submitted a mandatory comment in a text box to the oral presenter after viewing the oral presentation video, the next of the least viewed oral presentations would then be made available for peer review to the student in their BONGO Video Assignment folder, with a continuing of the availability of peer review of new oral presentations to peer review until the configured “Number of Required Reviews”, in our case 20 peer reviews of oral presentation videos, were met for every student.

A real advantage of a system that automatically populates the least watched video in BONGO Video Assignment folders is that it ensures all videos receive equal viewing and, importantly, when students drop out of the course, their oral presentation video also automatically drops out of the pool of available videos to peer review. In the end, we could give a student an oral presentation grade based on a relative ranking score of what 20 random students (~1/3 of enrolled students) felt that each oral presentation ranked in a pool of four oral presentations that each student peer reviewed every week.

3.4. Negative Aspects in Using the Bongo Video Assignment Tool within the Learn Course Website for Delivery of a 400-Level Asynchronous, Remote Learning Oral/Written Communication and Discussion Course

• Slow processing time of MP4 videos: Each uploaded video to BONGO Video Assignment folder needed to be up to twice as long as the video’s duration for narrated PowerPoint presentation videos exported as MP4 video file to be processed by the BONGO software. We were allowing oral presentations to be submitted up to 30 min in length, which meant that uploaded videos took up to an hour to process after uploading before they could be submitted as a video assignment. A minority of students reported poor internet connectivity and were unable to process their MP4 video files through the BONGO Video Assignment Tool. Solution: We had told students to go to a location where they could connect to a faster more reliable internet service. We recommended that students re-process their MP4 videos to minimize video file sizes using open-sourced video creation software, such as the VideoLAN Movie Creator (VideoLAN).
• Compatibility issues with exported MP4 videos from PowerPoint/Keynote within PC/MAC computers: If students used out-of-date PowerPoint/Keynote software, and older computers with out-of-date operating systems (especially MAC desktop computers/laptops), MP4 videos often would not be processed properly, and commonly the audio would be muddled or absent from the exported MP4 video of the narrated PowerPoint/Keynote file. Solution: A minority of students had to be forwarded to the IT Department and Computing Helpdesk to guide students through re-configuration and updates to their computers to ensure MP4 video conversion success.
• Videos which needed to be re-uploaded could not be repopulated to the same Video Assignment Folder: Students would peer review a processed MP4 oral presentation video through the BONGO Video Assignment Tool, only to realize that there were issues with it, such as the audio narration was absent or muddled. The instructor could manually “Reset Submission” within the BONGO Video Assignment Tool, which allowed the oral presenter to re-process and upload a newer, better version of their oral presentation video. However, once a video submission has been reset, it means the old uploaded video is removed from the pool of available videos, and any subsequent uploaded video re-enters the pool of available videos anew and is slotted in the queue to the next available student’s BONGO video assignment folder, requiring a peer review video below their peer review maximum of 20 videos. Because the videos are populated automatically in student Video Assignment Folders by BONGO, we were not able to re-assign the newer, updated oral presentation video to the student who had already been assigned and previously peer reviewed the older, previous version of the oral presentation video. It meant that this peer reviewer would have to evaluate an additional, random oral presentation video in lieu of the one that they had peer reviewed but was no longer in their available pool of videos to peer review.
• Concerns that students would pre-select the order in which they peer reviewed their allotment of 20 oral presentations. Students were to rank four oral presentations per week over five weeks, but BONGO Video Assignment Tool only permitted the setting of the number of required oral presentation videos that each student had to peer review in total, such as viewing of 20 videos, but we could not set limits on the access to oral presentation videos, which were fed to student BONGO Video Assignment folders on a weekly basis. This meant that students had access to view their quota of 20 peer review oral presentations all at once, and we could not configure BONGO Video Assignment Tool to restrict student peer review to a maximum viewing of four oral presentations per week. To prevent students from viewing all 20 videos and then choosing the four oral presentations out of the 20 in which to evaluate, we needed to increase the anonymity of the oral presentations so that students could not choose and select the order in which they viewed their oral presentations. Solution: We could choose “Conceal reviewer identity” and “Conceal submitter identity” under the Show Advanced configuration options to ensure that the identity of the submitter and the peer reviewer of the oral presentation video populated in every student’s BONGO Video Assignment Folder were anonymous. Since the icon of each oral presentation video was identified by a photo of the first slide, we required students to mask their video with an instructor-provided common, general first slide, with a subsequent, second slide template provided which contained identifying details such as the author of the oral presentation, an author-selected oral presentation title, and the instructor-provided unique research topic number and the associated title of the chosen research topic provided under the research topic group. This meant that students did not know of the identity of the oral presentations that they were going to view until after they had mouse-clicked on the oral presentation icon and started the oral presentation to view.
• Peer review comments left under the BONGO Video Assignment Tool under the “Automatic (System Selected)” peer review method are limited in visibility to the oral presenter, the reviewer, and the instructor: There needed to be peer reviews from 20 students for each oral presentation video, and each peer reviewer was required to submit at least one thoughtful and unique question to each oral presenter. Because of a configuration that limited private viewership of peer review comments to the oral presenter and instructor only within the BONGO Video Assignment Tool, we could not use the BONGO Video Assignment Tool for students to post their thoughtful and unique question(s), because as many as 19 different peer reviewers needed to be able to view the previous questions posed by students during previous peer reviews of an oral presentation. Solution: We had students post their thoughtful and unique question(s) outside the BONGO Video Assignment Tool, using the “Reply” function under each unique Discussion Thread containing the associated research abstract and appended PDF formatted files of relevant scientific papers for each research topic within the LEARN (D2L) course website. While our solution required students to jump between the BONGO Video Assignment Tool to watch their videos and to a Discussion Thread within the LEARN (D2L) course website to pose research questions in response to watching and analyzing the BONGO video, students quickly became accustomed to switching back and forth between the Discussion Threads within the LEARN (D2L) course website and the available videos provided within folders generated by the BONGO Video Assignment Tool. The Discussion Thread of every research topic became the hub of activity where research abstracts and PDF download of scientific papers were located. The students’ criticism towards this approach was that there were at least 20 questions posed by students under the Discussion Thread assigned to each research topic by week five of peer reviews, creating a long thread of responses, which students found irritating to scroll through.
• Formatting of the Video Assignment folder containing the oral presentations to “Automatic (System Selected)” meant that the oral presentation videos were private to select peer reviewers, and we needed to create an additional five open access “Use Manual (Learner Selected)” BONGO Video Assignment Folders, where students could populate video responses to peer review questions posed by the four students assigned to peer review their oral presentation video every week. The problem in having six different BONGO Video Assignment folders required for uploading of videos totaling many hundreds of videos for the class was that at least one or more students mistakenly populated their video in the wrong folder. BONGO lacks an overview function for the instructor to identify where a particular student’s video has been misplaced amongst the six different BONGO Video Assignment folders. Videos for every student are sorted under each Video Assignment Folder, but a general indexed search cannot be made by a student to identity which Video Assignment Folder a particular student had mistakenly uploaded a video to.
• Because peer reviews of oral presentations were populated randomly and privately to the inbox of select peer reviewers, this meant that each student had access to only 1/3 (20 out of ~60 students) of submitted oral presentations of the class. We should have had students upload an identical copy of their oral presentation video to a separate BONGO Video Assignment folder that was peer review formatted as open access: “Manual (Learner Selected)”, in addition to a separate Video Assignment folder that was configured to “Automatic (System Selected)” and limited in viewership to those required to review the oral presentation video. The open access: “Manual (Learner Selected)” formatted BONGO Video Assignment folder would have provided a repository for interested students to view oral presentation videos for everyone in the class, outside the 20 oral presentations that each student was required to peer review.
• A lack of integration of the BONGO Video Assignment Tool with instructor tools within the LEARN (D2L) course website meant that it was difficult to track ongoing student engagement and participation in the course. BONGO Video Assignment Tool randomly populated each student with 20 oral presentations to evaluate. Because of incompatibility of the BONGO Video Assignment Tool and instructor tools within the LEARN (D2L) course website, it was not easy to track whether students who had posed a question under a Discussion Thread within the LEARN (D2L) website corresponded to the appropriate oral presentation video which they were required to view within a BONGO Video Assignment Folder. Moreover, it was not easy to track a student’s video response submitted to a BONGO Video Assignment Folder addressing questions corresponding to questioners posed under a student’s Discussion Thread within the LEARN (D2L) website. For example, students would query to the instructor or two teaching assistants why they had more or less than the four students required to pose questions every week under their Discussion Thread within the LEARN (D2L) website. We were not easily able to retrieve this information for students. Data is not easily mined without cross-compatible, indexing functions between the 1308 posted questions in the Discussion Thread category within LEARN (D2L), with 2400 peer evaluations stored in the category of different online quiz folders in LEARN (D2L) and with the 363 video uploads populated in six different BONGO Video Assignment Folders.
• The appropriate location for students to upload their BONGO Video Assignments was counterintuitive. Students could not upload their video assignments in the LEARN (D2L) course site of what appeared to be labeled as the correct location to upload their video: > Submit > Video Assignment. “Submit Video Assignment” provided students with an overview of the titles of video assignments which were due, the type of project that was due, and a potential due date for the submission of video assignments. The real location to submit video assignments was an instructor created folder under a “Contents” subfolder on the LEARN (D2L) course website. Even though instructions for where to upload video assignments within LEARN (D2L) course website were made available in an uploaded HOW-TO video and in the contents of reminder emails to all students in the course, there were many students who contacted the instructor describing what they assumed was a unique problem in not being able to submit their assignments by means of the pull-down menu on the LEARN (D2L) course website, appropriately labeled “Submit Video Assignment”.

3.5. Overall Course Feedback from Students on Remote, Asynchronous Delivery of a 400-Level Oral Discussion/Presentation and Written Communication Course

3.5.1. Students Appeared to Prefer the Asynchronous, Remote Format for the Oral/Written Communication Course and Found the Course to Be of High Educational Value but Also a Lot of Work to Complete

Surprisingly, a majority of students who answered the course evaluation survey (see

Table 2. Course evaluation feedback from enrolled students on 400-level, remote, asynchronous delivery of an oral discussion/presentation and written communication course (n = 16).

#	Question	Student Responses (%)
1.	Would you recommend the remote and asynchronously delivered version of this 400-level oral discussion/presentation and written communication course for other students to take?
	I recommend the remotely and asynchronously version of this course for future students to take.	69.8%
	I DO NOT recommend the remotely and asynchronously version of this course for future students to take.	18.8%
	I AM NOT SURE if I would recommend this version of the course that I took	12.5%
2.	Would you have preferred to have taken this course as a “live” in-class Group Discussion course or do you prefer the remote participation approach?
	I would prefer to have taken this course as a “live” in-class Group Discussion course	31.3%
	I prefer the remote participation, asynchronous learning approach for this course.	50.0%
	I AM NOT SURE if I would recommend this version of BIOL 476 that I took	18.8%
3.	Did you have serious internet connection issues and/or MP4 formatting issues that hampered your ability to submit BONGO VIDEO ASSIGNMENTS on time?
	YES, I HAD serious internet connection issues and/or MP4 formatting issues.	25.0%
	NO, I DID NOT HAVE serious internet connection issues and/or MP4 formatting issues.	75.0%
4.	Did this 400-level oral discussion/presentation and written communication course require more, less or the same amount of work than other courses that you have taken?
	This 400-level course required MORE WORK than most other courses that I have taken	81.25%
	This 400-level course required LESS WORK than most other courses that I have taken.	6.25%
	This 400-level course required about the SAME AMOUNT OF WORK than most other courses that I have taken.	12.50%
5.	Are you more likely to consider continuing in research in a Masters or PhD graduate degree from having carried out a project involving research topic investigations and group discussions in this course?
	I am MORE likely to consider continuing in research in a post-graduate degree having taken this course	68.75%
	I am NO MORE likely to consider continuing in research in a post-graduate degree having taken this course	31.25%

The color and bold highlight the response most chosen by students in the questionnaire.

and Appendix M—Sample student feedback comments for the course) preferred delivery of this oral and written communication course by a remote, asynchronous format, and would recommend for future students to take the course in an asynchronous, remote format in the same manner that they did, rather than to enroll in the same course offered as a standard “Live” in-class, oral presentation, and group discussion course.

The students’ preference for a remote, asynchronous, delivery of this communication course appears to largely relate to the comfortability of students in being able to deliver a narrated oral presentation without the anxiety in having to perform in front of a live audience. Moreover, the asynchronous delivery format gave students many days to upload a video response to questions posed of their oral presentation. The asynchronous delivery format provided students with time in which to research and prepare thoughtful answers to questions posed to them, without feeling intimidated to respond to questions unprepared and under the stress of a live audience.

What these students were not able to judge in their assessment of the remote, asynchronous learning format is the absence of instructor input in the remote learning format. A professor is more able to facilitate, monitor, and guide student learning and discussion within the in-class “live” version of the course. Some students left course feedback wishing that they had received more guidance in how to research their chosen topic, and in communicating scientific research, both orally and written. While many students expressed in anonymous feedback of the course how much they appreciated the self-learning opportunities in the course, and the peer engagement in a wide variety of research topics, other students found that this more self-learning approach does not suit their learning style.

Most students who answered the course feedback survey felt that the course was in the category of “more” or “most” difficult of the courses that they were taking at the same time, with almost all believing the course entailed a “heavy” to “heaviest” workload compared to the other courses they were taking. While complaining about the heavy workload of the course, a student did remark that they did not “realize just how much [that they had] learned” until the course was over.

Students who opted to submit course feedback were nearly unanimous in their statements about the “high” or “very high” educational value of the course, and felt that they were more interested in carrying out research in a Master’s or PhD graduate degree after having carried out an individual research project investigation and having contributed to research discussions in this course.

3.5.2. Students’ Approval of Online Learning and the Asynchronous Delivery of a Communication Course Is Consistent with Outcomes Reported by Many Studies

There are reported benefits and drawbacks of online learning [10,11], and the degree of success appears to relate to how well the technology is employed to enhance engagement and interactivity, over in-person learning opportunities for enrolled students [12,13]. Having students carry out preparatory learning on their own through research, reading, and watching videos, and then having them engage later to discuss what they have learned in a group classroom setting, whether in-class or virtually, creates the so-called “flipped classroom” that can result in significant improvement in student learning compared with traditional teaching methods [14,15,16].

The use of asynchronous videos for the virtual classroom appears to work best when: (a) there are strict guidelines and deadlines that drive student interactivity, (b) when the instructor facilitates but does not dominate the classroom discussion, and (c) the students receive regular feedback on their class performance during the school term [17,18,19,20,21]. A general consensus is that course content created by student-produced video assignments generates more student engagement and learning opportunities than more passive learning in watching and learning from instructor created videos [22,23,24].

3.5.3. Some Students Encountered Serious Problems Converting Their Narrated PowerPoint Presentations into Exported MP4 Videos, and Processing Uploaded MP4 Formatted Videos within the BONGO Video Assignment Tool

Outside complaints of the self-learning approach not suiting every student’s learning style and the workload burden of the course, another category of major complaint towards the course involved formatting issues of MP4 files and time for uploading and conversion of videos using the BONGO Video Assignment tool. One quarter of students who answered the course feedback survey reported having serious internet connection issues and/or MP4 formatting issues that hampered their ability to submit BONGO Video Assignments on time.

3.5.4. Students Were Understandably Fearful of a Peer Review Ranking System of Their Oral Presentations

Many students expressed a hesitancy in that 35% of their final grade was being assigned through a peer ranking system of their oral presentations, which did not involve instructor authority over assigning of grades. Students take the peer ranking system seriously and were highly consistent in their choice involving which of the oral presentations they ranked highly and those which they assigned low ranking scores.

For example, one of the highest ranked oral presentations was ranked “Best” (11 times), “Second Best” (five times), “Third Best” (two times), and “Least favorite” (one time). This is a student who was consistently ranked “Best” or “Second Best” oral presentation by 84.2% (that is 16 of 19) of different peer reviewers of the four oral presentations which students viewed in a week.

One of the lowest ranked oral presentation was ranked “Best” (zero times), “Second Best” (one time), “Third Best” (six times) and “Least favorite” (11 times). This is a student who was ranked in the bottom two categories “Third Best” or “Least favorite” oral presentation by 100% (that is by 17 of 17) of different peer reviewers of the four oral presentations which students viewed in a week.

Because of minor differences in the number of reviewers received for each oral presentation, ranging from 16 to 20 reviewers (average = 17.6), ranking scores were tabulated as a summation of scores in each of the four categories (“Best” “Second Best” “Third Best”, and “Least favorite”) normalized as a fraction over the total number of reviews which each student received for their oral presentation.

3.5.5. Despite the Fearfulness of Students in Use of a Peer Review Ranking System, There Was a Strong Correlation Between the Student Peer-Graded Work and the Independently-Graded Instructor-Graded Work

The student peer review rankings of oral presentations (worth 35% of the final grade) were scaled to a mean student score of 74.25 +/− 1.96%, with a minimum score of 38.6% to a maximum score of 100% for n = 60 enrolled students (Figure 1a). The instructors and teaching assistants did not have time to view and evaluate each of the videos uploaded by students, given the total volume of uploaded videos submitted by students. There were >363 total video uploads by students consisting of oral presentation videos (63 narrated Powerpoint presentation videos of up to 30 min in length each, totaling up to a maximum of 30 h of oral presentations), as well as >300 videos submitted of responses to student questions of each peer viewed oral presentation (of different video lengths).

The independently graded written work marked by the instructor (worth 35% of the final grade) consisting of a submitted research abstracts, research reports on students’ chosen research topics, and essays on the research topics provided to them (each piece of written work was 15% of the final grade). The frequency histogram illustrating the spread of grades reveals a less than normal distribution of instructor marked grades (Figure 1b), compared to the student peer reviewed grades (Figure 1a), with a tendency of scores clumped into a very high bin (85–90%) resulting from many students’ strong written work submitted in the excellent category (16 students) and a clumping of grades in the lowest possible bin (55–60%) (10 students), in the barely acceptable range of what was poorly written submitted work. While the instructor-graded work did not follow a typical normal distribution of grades, the mean +/− s.e.m. = 74.9 +/− 1.6% of instructor-graded work (Figure 1b) was nearly identical to the student peer graded work (mean +/− s.e.m. = 74.3 +/− 1.9%). The calculated Pearson’s Correlation Coefficient (R value) indicates a strong correlation of 0.77, between the student peer review graded work and the instructor-graded work (Figure 1c), despite the fact that the instructor marked the written work without the instructor having examined the submitted narrated oral presentations, or submitted video responses to peer questions, or knowing the peer ranking of student oral presentations when marking the written work. This suggests that the students who tended to be ranked favorably by student peer review were also the students who tended to submit the strongest written work for the course. The strong correlation between the student peer review graded work and the instructor-graded work indicates that the students were able to objectively evaluate oral presentations for their scholastic merit, closely corresponding to how the instructor independently assessed the depth and strength of the student’s research, as revealed in the submitted written work.

4. Conclusions

While a remote, asynchronous learning approach is non-ideal, and will never replace an instructor who is more able to directly facilitate learning and discussion in a “live” classroom with small enrolments (<15 to 25 students), there were (to our surprise!), real advantages in delivering an oral/written communication course remotely and asynchronously.

(1) Student accessibility is the greatest advantage of a remote, asynchronous learning delivery of an oral/written communication course. Normally much of in-class time is devoted to the “live” delivery and discussion of each student’s oral presentations, one-by-one, to a maximum of 25 students, staggered over the last six-weeks of the course. In the remote, asynchronous delivery format, we were able to have all of the ~60 enrolled students bank their narrated oral presentations in video format by the same deadline date at the end of the first half of the course by Week #6. Then in the last half of the course (Part B), we were able to set instructions to enforce engagement and peer evaluation of enrolled students into a total of 1200 unique virtual classrooms consisting of an oral presenter and peer reviewers of oral presentations every week, over a time span of five weeks.

(2) Asynchronous and remote delivery of the course meant that students could deliver their oral presentation and respond to peer questions of their oral presentation at their leisure and convenience, without undue stress and anxiety associated with delivery of a “live” performance in front of their peers.

(3) In a standard in-person class, we would have time set aside over a few minutes for a few students to field questions to an oral presenter (in what was usually an awkward exchange between students because of the typical nervousness of students) in a “live”, oral performance of student presentations in class. In the remote, asynchronous format, each student could receive unique and thoughtful questions from 20 different students who viewed their narrated oral presentation video. Additionally, since it was a process where students responded to these 20 unique, thoughtful questions posed by students stretched over a period of five weeks, it provided opportunities for each student to steadily increase the depth of understanding of their own research topic through sequential peer engagement over a five-week period after they needed to research, prepare, and upload their narrated oral presentation onto the course website.

(4) A remote, asynchronously delivered course can facilitate much more peer communication than in a “live” oral presentation/discussion course. There was a total of 62 oral presentations and 301 uploads of video responses to 1985 questions posed by peers over the 12-week course. There was a total of 1159 feedback questionnaires and 1066 student rankings submitted for peer review of oral presentations, and 1091 evaluations submitted by oral presenters evaluating the quality of questions posed by reviewers of oral presentations.

(5) A drawback in the remote, asynchronous delivery of the course is that most of the peer engagement is blind to the single instructor and two teaching assistants, based, in large part, on the enormity of the peer engagement through the internet and course website portal, consisting of hundreds of hours of uploaded student videos and thousands of questions posed by enrolled students and thousands of submitted peer evaluations. Compounding the issue is the inability of instructors or the two teaching assistants to cross-index data linked between the student entries in the LEARN course website and the uploaded videos stored within the BONGO Video Assignment tool. The weighting of peer evaluation of oral presentations and quality of questions posed of oral presenters was limited to 35% of the total grade for the course, since the instructor and teaching assistants were mostly blind to the depth and quality of the peer engagement and evaluation of oral presentations.

The consistent engagement of students, and the positive feedback from enrolled students in the course, despite its remote, asynchronous delivery, indicate that a future version of this oral/written discussion course continue to be delivered remotely and asynchronously, in whole or in part, even without a mandated delivery of the course by a remote and asynchronous method due to the COVID-19 pandemic restrictions in 2020–2021.