Evaluating a Structured Online Peer Evaluation System Among Graduate-Level Communication Capstone Students Through Action Research

Although enrollment in online courses continues to accelerate, challenges exist in online learning. A failure to experience collaboration and interaction can impact student retention and success. While peer review activity promotes student interaction, a collaborative community of learners, and critical thinking skills, higher education environments have failed to equip students with the knowledge and tools to ensure adept participation. As students offered limited participation and low-quality engagement in routine online peer review activities, the purpose of this action research was to implement and evaluate the impact of a structured online peer evaluation system for Graduate Communication Capstone students at the University of North Coast Muscari (UNCM). This study incorporated a structured peer evaluation system, including an interactive educational technology peer review tool kit innovation. The theoretical framework of the innovation was aligned to learning theory and grounded in Vygotsky’s zone of proximal development, cognitive and mind tools, and Constructivist theory of cognitive apprenticeship. Data collection offered seven methods and data analysis included quantitative and qualitative approaches as part of a triangulation mixed methods design. Community of Inquiry (CoI) deductive analysis was performed to denote social and cognitive presences, while further validating the themes that had emerged through qualitative data analysis. As an impact of this research study, students used the structured peer evaluation system to transform anxiety into social and cognitive freedom, producing a focused, responsible approach to peer learning.

Enrollment in online courses continues to accelerate (Hart et al., 2021;Picciano, 2019) as the use of web-based technology continues to extend "the boundaries and pedagogies of teaching and learning" (Cheng & Chau, 2016, p. 257).In the tenth annual report of Changing Course: Ten Years of Tracking Online Education in the United States, the rate of online enrollments far exceeded those across higher education overall (Allen & Seaman, 2013).In 2016, 72% of public universities and 50% of private, non-profit educational institutions offered completely online programs (Xu & Xu, 2019).In research conducted in January and February 2020, more than half of online college students noted that if their online programs became unavailable, they would seek a comparable online program as on-campus enrollment was not an option (Magda et al., 2020).Of those students surveyed, one-third expressed a desire to take additional online courses following their degree completion (Magda et al., 2020).
However, there are challenges to success in the online learning environment.Engaging students in online learning is not an easy endeavor.Regular participation frequently involves a small number of students while others wait and engage very little or not at all (Barría et al., 2014).This difference in interaction relative to face-to-face courses can lead to feelings of isolation for learners (Negash, 2008;Yuan & Kim, 2014).The failure to experience collaboration and a lack of interaction are among the factors impacting student retention and success in the online environment (Conrad & Donaldson, 2004;Heyman, 2010;Lee & Choi, 2011, Willging & Johnson, 2009).
Despite the various merits of online learning, the lack of physical presence and face-toface interaction can offer the absence of spoken and visual cues (Alman et al., 2012) and cause students to suffer from feelings of loneliness and inadequate social engagement (Purarjomandlangrudi et al., 2016).As participation is an inherent factor of learning (Wenger, 1998), its importance is paramount.In a study that examined the correlation between online participation and grades, those students who failed one or more of the learning modules interacted less often than peers who attained passing grades (Davies & Graff, 2005).In turn, an elevated level of student participation and activity has the potential to offer a positive impact on academic achievement and deliver a stronger e-learning experience (Cheng & Chau, 2016;Huang et al., 2012;Michinov et al., 2011).
The peer review process has many benefits and is an important tool in online higher education learning environments.During peer review, students employ critical thinking skills (Demirbilek, 2015;McMahon, 2010), gain insight into different perspectives (Hogg, 2018), and engage writing and organizational skills (Man et al., 2018).Most important, peer review provides the opportunity for student interaction and collaboration within the online environment and encourages the development of a community of learners (Moneypenny et al., 2018).
Even so, to reap the benefits of peer review, students must choose to actively take part.Although peer review is lauded as an effective, collaborative online tool that allows students to experience analysis, synthesis, and evaluation processes (Demirbilek, 2015;Li et al., 2010;Lynch et al., 2012), higher education environments fail to equip students with the knowledge and tools related to peer review assessment (Nicol et al., 2014).Specifically, students do not receive sufficient preparation and training to formulate and deliver feedback to their peers, nor do they receive guidance on how to interpret the feedback received (Nicol et al., 2014).For peer review to be a successful learning opportunity, online students must receive strong guidance on how to fully participate and become actively engaged in the process.

Literature Review
The review of literature includes conceptualizing peer review through theoretical alignment, advantages and disadvantages of peer review, and an examination of peer review tools and methods.

Theoretical Alignment to Peer Review
Peer review offers an interactive experience through which knowledge is constructed collaboratively.In turn, peer review aligns to the learning theory of constructivism as per John Dewey (1916Dewey ( , 1938)): Constructivism is not the act of telling or being told, but a constructive process.As opposed to knowledge that is passed from instructor to learner through rote memory, constructivism provides for the creation of knowledge through experience (Dewey, 1938;Ertmer & Newby, 2013;Jaramillo, 1996) and through contexts that have the capacity to enhance student learning (Biggs, 2011).
In alignment with the social constructivist theory of learning, peer review provides a collaborative culture of learning.Vygotsky's (1962) social constructivist theory of learning claims that students' skills and knowledge are shaped through cultural interaction.Learning becomes a social activity where learners interact and cognitive growth is stimulated (Schunk, 2008).
During peer review activities, participants experience the attributes of the constructivist theory of cognitive apprenticeship.Students are able to learn through observation, imitation, and modeling (Collins, 1988;Collins et al., 1987).Correspondingly, the methods dimension of cognitive apprenticeship seeks to adapt student behaviors into genuine practices through activities and social engagement opportunities (Brown et al., 1989).

Advantages and Disadvantages of Peer Review
Although peer review is often heralded for the benefits it provides, research findings indicate that there are perceived advantages and disadvantages to its implementation.
Through meaningful and active engagement in peer review, students offer inquiries, deliver positive commentary, and identify areas for improvement (Ching & Hsu, 2016;Gikandi & Morrow, 2016).By way of shared perspectives and offers of feedback and guidance, students move from hesitation to active engagement within a robust learning community (Dar et al., 2014;Gikandi & Morrow, 2016;Kearney, 2013).In addition, the exchange of information during peerto-peer feedback allows students to increase comprehension and learn new approaches through exposure to different perspectives (Demirbilek, 2015;Gikandi & Morrow, 2016;Hogg, 2018).
When learners are aware of an upcoming peer review task, they can offer increased motivation and care in the preparation of their work (Dar et al., 2014;Llado et al., 2014).In interdisciplinary research by Llado et al. (2014), university students reported that peer assessment prompted them to take additional time to prepare stronger work.Therefore, peer review serves as an effective strategy to prompt students to plan ahead, engage in formative feedback, and revise work prior to final submission (Baker, 2016).

Persistent Issues and Concerns
While research findings indicate numerous advantages to peer review, issues and concerns remain.Frequently, students admit that it can be difficult to critically assess the work of peers (Demirbilek, 2015;Llado et al., 2014;Mulder et al., 2014) due to friendships and the potential for damaged relationships (Hogg, 2018;McMahon, 2010).For example, undergraduate students at a New Zealand university reported concerns over the fairness of peers' assessment, stating that established relationships made it harder to critique than to deliver praise (Hogg, 2018).
When students associate limitations, distaste, or low value with peer review, their motivation to participate may diminish, and they may resist engagement (Brill, 2016;Wang, 2016;Zong et al., 2022).Even when students receive proper peer review training, some students may not take peer review seriously and consider it to be unrealistic and a waste of time (Dar et al., 2014).
Students can experience anxiety and intimidation as they consider the level of responsibility and the amount of time required to mark the work of their peers (Llado, et al., 2014;Moneypenny et al., 2018).In research by Nagori and Cooper (2014), postgraduate students acknowledged questioning both their peer review abilities and those of their classmates, reporting that it had been an unsettling experience.Furthermore, students share their concern about peers reviewing their work and observing their weaknesses (Dar et al., 2014;Llado et al., 2014).

Peer Review Tools and Methods
Research indicates that there are opportunities to utilize peer review tools in support of the processing and management of peer review activities (Caddy, 2014;Mulder et al., 2014;O'Connor & McGuigge, 2013;Sridharan et al., 2018).PRAZE, an electronic peer review management tool, was reported to be useful in distributing articles to ensure that each article received multiple reviews (Mulder et al., 2014).Similarly, in undergraduate research by Caddy (2014), the online tool SPARKPLUS recorded a high level of group peer review engagement and delivered a reduction in social loafing.
The integration of scripts and prompts can assist students in creating feedback and serve as a framework for analysis (Ching & Hsu, 2013, 2016;Nicol et al., 2014).In addition, exemplars and guides, such as instructional procedures for peer assessment, can prove beneficial for leading and directing students in their review of peer work and in the creation of feedback (Brill, 2016;Dar et al., 2014;Nagori & Cooper, 2014;Wang, 2016).In research by Reinholz (2018), the use of reflective questions, checkboxes, and hints was reported to offer guidance for students.Furthermore, research involving graduate instructional design students suggested the need to support peer review efforts through scaffolding and ample resources, such as checklists and models (Brill, 2016).
Numerous opportunities exist for peer review activities within the online course design, software, and Learning Management System (LMS) of higher education institutions (Gikandi & Morrow, 2016;Hampel & Pleines, 2013;Nicol et al., 2014).By creatively utilizing the asynchronous discussion forums, students can post and share their work for active conversation and collaboration (Gikandi & Morrow, 2016).Furthermore, institutions may choose to select external peer review environments to entice users with well-known, popular settings.Research asserts that wiki sites, Facebook, and Twitter are compelling platforms for social and collaborative peer learning (Demirbilek, 2015;Evans, 2015).

Research Purpose and Direction
The existing Graduate Communication (GRAD COM) Capstone environment at the University of North Coast Muscari (UNCM) (a pseudonym) lacked a structured online peer evaluation system with effective peer evaluation tools to prepare students for peer assessment, promote peer review participation, and ensure that students received the benefits associated with peer review, whether giving or receiving feedback.Students offered limited participation and low-quality engagement in routine online peer review activities and until the dilemma was fully addressed and rectified, it was assumed that peer review participation would remain low.Therefore, two primary research questions guided this action research study.

Research Questions
1. How does using a structured peer evaluation system impact the peer review process in an online Graduate Communication Capstone classroom at UNCM?
2. What are the perceptions of students regarding a structured peer evaluation system in support of online asynchronous peer review activity in a Graduate Communication Capstone classroom at UNCM?

Method
This action research study was conducted at the College of Online and Continuing Education (COCE) at UNCM.The private nonprofit university, which currently enrolls over 135,000 students, hosts over 200 programs.The research took place in the GRAD COM Capstone classroom via the online Brightspace Desire to Learn (D2L) LMS.Study participants included a convenience sample of students participating in their final course in support of an MA in Communication degree.
Of the 14 Capstone students who received the UNCM IRB Consent Form as an invitation to participate in the study, seven students signed the IRB Consent Form and consented to study participation.All seven study participants participated in the preterm and post-term questionnaires with six of the seven students participating in one-on-one interviews.Additional demographic information about participants was not able to be gathered and reported due to UNCM IRB restrictions.

Innovation
An interactive peer review tool kit was created as part of the structured peer evaluation system for this study.The innovation offered foundational alignment to learning theory and was designed to promote participation and empower students to engage and provide feedback at a higher-quality level.As students can feel detached from dialogue and direction in the online classroom, the tool kit allowed the instructor to provide access to helpful resources so that students could determine which ones worked best for them (Schrenk et al., 2021).
In alignment with Vygotsky's (1978) work with students of similar mental development and their ability to handle problems independently up to a certain level of difficulty, all GRAD COM students were positioned to enter the Capstone course with similar course and credit hour profiles.In turn, the Capstone innovation was positioned to elevate students of similar standing from independent problem-solving levels at the lower end of the zone of proximal development to a higher level of knowledge (Vygotsky, 1978).This was accomplished through the provision of scaffolding, guidance, and support provided through the expertise of a more knowledgeable other (Vygotsky, 1978).
The use of cognitive and mind tools in education is represented through computer programs, applications, and technology that allow users to participate in higher-order learning and enable critical thinking skills (Kirschner & Erkens, 2006).In turn, the innovation for this research study provided access to a collection of computer-based cognitive tools which could be used to create and facilitate technology-enhanced dialogue, extend learning, and further enhance collaboration (Kirschner & Erkens, 2006).
In alignment with the Constructivist theory of cognitive apprenticeship, the innovation design was further influenced by the concepts of modeling, coaching, scaffolding, articulation, reflection, and exploration (Collins et al., 1987).The interactive elements of the innovation were grounded in research and aligned with the cognitive apprenticeship components (see Table 1).

Figure 1 Responsive Design of the Innovation in the Structured Peer Evaluation System
The innovation served as a repository for eight learning modules and their supporting cognitive tools.The modules included (1) Learning Module Options, (2) Sixty Seconds of Knowledge: Video Clips, (3) Navigating the Peer Review Process: Support Tools, (4) Interactive Learning Activities, (5) Getting Started: Questions & Prompts, (6) Final Project Rubric Reminder, (7) Reflection: Practice & Self-Check, and (8) Exploration: Independent Learning.

Data Collection Methods
To fully examine the proposed research questions, seven data collection methods were utilized (see Table 2).The data sources included a preterm questionnaire, a post-term questionnaire, post-term questionnaire open-ended questions, observational field notes, one-onone interviews, researcher's handwritten interview notations, and student post artifacts.

Table 2 Research Questions and Data Sources
Research Questions Data Sources RQ1: How does using a structured peer evaluation system impact the peer review process in an online Graduate Communication Capstone classroom at UNCM?

Preterm Questionnaire
The preterm and post-term questionnaires for this study were constructed from two published survey instruments.Questions 1-10 of the instrument (Part One) were based on Kaufman and Shunn's (2011) research survey and were positioned to evaluate students' perceptions regarding online peer assessment.The remaining 20 questions (Parts Two and Three) of the instrument were created based on research by Moneypenny et al. (2018) that aligned specifically with Wen and Tsai's (2006) four subscales of peer review.The subscales within the questionnaires were referenced as (1) Positive Attitude Subscale (POS), (2) Online Attitude Subscale (OAS), (3) Understanding-and-Action Subscale (UAS), and (4) Negative Attitude Subscale (NAS).
The purpose of the preterm questionnaire was to gauge students' perceptions of the existing peer review process at UNCM or their former participation in peer review activity (see Appendix A).During the two-week period prior to the term kick-off and following UNCM Institutional Review Board (IRB) approval, the Capstone students received a UNCM email with the Capstone Peer Review IRB Consent Form as an invitation to participate in the study.Students who signed and submitted the consent form prior to the beginning of the Capstone term were eligible for study participation and received a follow-up email with a link to the quantitative questionnaire, housed in SurveyMonkey.Due for completion before the start of the term, the questionnaire offered 30 questions based on a five-point Likert scale.

Post-term Questionnaire
A final quantitative questionnaire, positioned to measure the usefulness of the research intervention (Creswell, 2014), mirrored the three sections outlined in the preterm questionnaire.Immediately following the conclusion of the term, study participants received a UNCM email with a link to the post-term questionnaire, located in SurveyMonkey (see Appendix D).The study participants received three weeks to complete the post-term questionnaire.

Post-term Questionnaire Open-ended Questions
A fourth section was added to the post-term questionnaire and focused specifically on the structured peer review innovation.Thus, Part Four offered a qualitative component of the postterm questionnaire, consisting of six open-ended questions that students completed as part of their response to the post-term questionnaire (see Appendix D).

Observational Field Notes
Following the term conclusion, the researcher recorded observational field notes to describe the interactivity of the peer review participants (see Appendix B).The notations included posting patterns and additional collaborative activity deemed to be significant.As the Capstone class size was small, the observations provided an opportunity to gather data on actual student behaviors instead of relying solely on students' self-reported feelings and perceptions (Schmuck, 1997).

One-on-One Interviews
In alignment with UNCM IRB requirements, qualitative one-on-one interviews were conducted following the term conclusion.The purpose of the 20-25-minute semi-structured interviews was to question participants about their experiences with the structured peer evaluation system and the peer review tool kit (see Appendix C).The interviews yielded direct quotes from participants and offered insight into their opinions and experiences (Patton, 2014).

Researcher's Handwritten Interview Notations
During each of the one-on-one interviews, the researcher recorded handwritten notes of impressions and interesting aspects as they surfaced (see Appendix C).Interviews were approached through in-depth inquiry to ensure that the research topic was fully discussed and documented in support of potential changes to current systems (Patton, 2014).

Student Post Artifacts
Student post artifacts were created within the discussion board forum of D2L Brightspace by way of student interaction during the term.As study participants provided original and response posts during the peer review activities, conversational threads developed.These student post artifacts remained within the Capstone course environment during and after the study term and were later collected for CoI assessment (Garrison & Arbaugh, 2007).

Data Analysis
The data analysis process for this study embodied methodological techniques to analyze the data and to ensure that the information provided alignment to the study's research questions (Mertler, 2017).

Quantitative Data Analysis
The use of preterm and post-term questionnaires provided the opportunity to measure and produce numeric data.
Cronbach's Alpha.Prior to calculating the descriptive statistics for the preterm and post-term questionnaires, the reliability, or internal consistency, of the two instruments was assessed by calculating Cronbach's alpha for each part of each questionnaire (Cronbach, 1951;Tavakol & Dennick, 2011).Through this interpretation of reliability, Cronbach's alpha offered insight into the inter-item relationship of the questionnaire parts and how well the items correlated and measured the same characteristics (Tavalok & Dennick, 2011;Roever & Phakiti, 2018).
Descriptive Statistics Analysis.To evaluate the quantitative results from the two questionnaires, descriptive statistics analysis was utilized to "summarize, organize, and simplify" (Mertler, 2017, p. 178) the data.
Shapiro-Wilk Test and the Wilcoxon Signed Rank Test.To test the normality of the data and to determine if the data were normally distributed for the population, the researcher conducted a Shapiro-Wilk test in JASP (Version 0.11.0;2020), an open-source statistical software program supported by the University of Amsterdam.Although a deviation from normal was not indicated by the Shapiro-Wilk test (Shapiro & Wilk, 1965), the Wilcoxon signed rank test, a non-parametric test, (Wilcoxon, 1945) was run, due to limited data for the seven study participants.The Wilcoxon signed rank test was conducted for each part (Part One, Part Two, and Part Three) of the questionnaires to assess whether the mean scores from preterm questionnaire to post-term questionnaire differed significantly (Wilcoxon, 1945).The utilization of an alpha value of .05allowed the researcher to ensure with reasonable certainty that only 5% of the time would the differences attained actually be because of chance or sampling error (Johnson, 2008;Mertler, 2017).Results with a p value of less than .05were statistically significant.
Bonferroni Adjustment Test.As more than one questionnaire part was aligned to one research question, the Bonferroni adjustment (Streiner & Norman, 2011) test was run to verify if each questionnaire part was independent of each other.To produce a significant result, it was necessary for the Bonferroni adjustment test to produce a p value of less than .017(Streiner & Norman, 2011).

Qualitative Data Analysis
The qualitative data for this study yielded vast amounts of unstructured data; however, through qualitative analysis, the masses of text were brought into a more meaningful form and framework (Yee, Wong, & Turner, 2017).To reduce the amount of qualitative data collected, the researcher used inductive analysis (Mertler, 2017), as well as CoI analysis with a priori categories for social and cognitive presence (Garrison & Arbaugh, 2007;Van der Merwe, 2012).
Inductive Analysis.To make sense of the qualitative data compiled from the observational field notes, the one-on-one interviews, the researcher's handwritten interview notations, and the post-term questionnaire open-ended questions, the data were segmented, taken apart, and put back together (Creswell, 2014;Flick, 2009).The ultimate goal was to reduce the qualitative information into patterns and themes for a representation of the research discoveries (Johnson, 2008).Once all data sets were organized and prepared, inductive analysis proceeded on two levels.First, a handwritten memoing process was conducted, followed by computer-aided analysis.
Computer-aided Analysis.Digital content from the four data sources was uploaded into Delve, an online digital tool for creating projects and coding digital transcripts ("Delve," n.d.).Coding began with Structural Coding to align the segments of data with the study's research questions (Saldaña, 2016).Thereafter, a second round of Descriptive Coding and a third round of Process Coding (Saldaña, 2016) were conducted.As a fourth and final round of first cycle coding, In Vivo Coding was conducted on all data sources except for the observational field notes as they did not represent the voices of study participants (Saldaña, 2016).Supporting analytic memos were created to offer a description of each code.
Next, in seeking to discover categories, the researcher moved from the Delve coding environment back into Microsoft Word and organized and assembled the first cycle codes through a code mapping process.During code mapping, codes were organized and visually displayed (Saldaña, 2016).During a second iteration of code mapping, the researcher reviewed the codes and began to assess, organize, and group the codes (Saldaña, 2016) until ten categories emerged.
Identification of Themes and Presentation.Upon completion of code mapping, a second cycle approach was utilized to reduce data into smaller units (Saldaña, 2016).The researcher transitioned from Microsoft Word into the physical environment where foam core boards were used to pin, move, and rearrange the codes by category.The researcher utilized the categories that had been created in a second iteration of code mapping and through pattern coding to group the original codes by pattern.Analytic memos were created for each of the ten categories.The analysis proceedings continued to evolve as the researcher sought to link categories and identify emerging themes and patterns (Clark & Vealé, 2018;Esterberg, 2002).Ultimately, three themes were identified to communicate study participants' experiences and behaviors (Saldaña, 2016).
Community of Inquiry Analysis.A fifth qualitative data set was generated through student peer review posts and responses provided during the active term.For qualitative analysis purposes, student posts were treated as course artifacts as they were the tools "to get work done" (Saldaña & Omasta, 2017, p. 74) during peer review.Once again, data analysis began with a general approach followed by a computer-aided approach.Printed copies received initial memoing and highlighting based on the seven established a priori category codes for social and cognitive presence (Garrison & Arbaugh, 2007;Van der Merwe, 2012).
Next, the researcher returned to Delve and created a separate project distinct from the previous inductive analysis project.Student post artifacts were uploaded into Delve as separate transcripts for Week Four and Week Seven, after which seven codes were created in Delve to align with Garrison and Arbaugh's (2007) a priori category codes for social and cognitive presence.Finally, supporting analytic memos were created and aligned.
Based on Garrison and Arbaugh's (2007) CoI categories and presence indicators, a sentence-by-sentence analysis was utilized with social presence coded first, followed by cognitive presence.Moving forward, the researcher tallied CoI categories and indicators in Delve, entering totals and percentages into an Excel spreadsheet.

Integration
Through a triangulation mixed methods approach, both quantitative and qualitative data were evaluated.The findings of the two analyses were integrated via a convergent process to provide a more comprehensive review of the research topic (Mertler, 2017).

Results
For this study, quantitative and qualitative data were collected in a mixed methods approach and analyzed through triangulation to corroborate the findings (Lincoln & Guba, 1985).Through confirmation of multiple processes, the certainty assigned to data interpretation was increased (Webb, et al., 1966).Triangulation ensured that the flaws of one process were "cancelled out by the strengths of another" (Lincoln & Guba, 1985, p. 306).

Summary of Quantitative Findings
Quantitative data collected in this study included study participants' feedback from a preterm questionnaire and a post-term questionnaire.During quantitative data analysis, Cronbach's alpha was calculated for each part of the preterm and post-term questionnaires, offering low and varied internal consistency.Descriptive statistics were calculated for each part of each questionnaire.Although a deviation from normality was not detected in the Shapiro-Wilk test (Shapiro & Wilk, 1965), the Wilcoxon signed rank test was run due to the limited number of study participants.Results from the Wilcoxon signed rank test and the Bonferroni adjustment test produced no statistically significant results.

Summary of Qualitative Methods and Findings
In this study, qualitative data was collected from five data sources including six post-term questionnaire open-ended questions, observational field notes, one-on-one interviews, researcher's handwritten interview notations, and student post artifacts.

Inductive Analysis Results
First cycle and second cycle coding of the first four data sources produced ten categories and three qualitative themes.The themes included Theme I: Comprehensive peer review tool kit promoted student confidence and empowerment, Theme II: Peer review engagement fostered appreciative, collaborative community of learners, and Theme III: The structured peer review system transformed student anticipation and anxiety into a focused approach to learning.

Community of Inquiry Findings
CoI coding of the fifth data set, student post artifacts, was conducted separately.During CoI coding, a total of 598 codes were applied across 24 student threads.Using the seven a priori CoI categories and performance indicators (Garrison & Arbaugh, 2007), 349 occurrences of social presence and 249 occurrences of cognitive presence were recorded (see Table 3).Through triangulation, the researcher corroborated the qualitative themes and the CoI findings to test for rigor (Lincoln & Guba, 1985) and to provide an increased assurance for the meaning of the data (Webb et al., 1966).The results were further valdidated through alignment to study participant (referenced by pseudonym) examples and existing research (see Table 4).• During open communication with peers, the study participants demonstrated confidence and a sense of ownership for their comments.
• Empowered by the structured peer evaluation system and more specifically, by the resources and tools shared within the peer review tool kit, study participants displayed a freedom to engage with peers.
• Students displayed a sense of comfort and self-confidence with the process and become more effective as peer assessors (Brutus et al., 2013).
A structured peer evaluation system can be utilized to "promote, facilitate, and standardize" (Brutus et al., 2013, p. 18) Vygotsky (1962) proclaims that students' skills and knowledge are shaped through cultural interaction.
Learning becomes a social activity in an environment where learners interact and where cognitive growth is stimulated (Schunk, 2008).

Discussion
To fully understand the results from this study, it is important to situate and interpret the findings within the research questions and in alignment with the voices of the study participants (referenced by pseudonyms).Research Question 1: How Does Using a Structured Peer Evaluation System Impact the Peer Review Process in an Online Graduate Communication Capstone Classroom at UNCM?
The findings from the converged observational field notes and the student post artifacts revealed that students assumed a responsible role in the construction of collaborative learning.The student post artifacts, displaying student peer review engagement in Week Four and Week Seven, reflected the study participants' use of the structured peer evaluation system to trigger their active participation.Moreover, students were prompted to express themselves both socially and responsibly and to openly share cognitive knowledge with peers.

Students Assumed a Responsible Role in the Construction of Collaborative Learning
The structured peer evaluation system was designed to empower students to take on a responsible role during peer assessment as they constructed new meaning during the evaluation of peers' work and produced an interpretation and feedback based on their individual experiences, beliefs, and thought patterns (Jaramillo, 1996;Jonassen, 1991;Powell & Kalina, 2009).The goal was to empower students through training, resources, and tools.As opposed to rote learning during which knowledge is simply passed from instructor to student, the learning theory of constructivism (Dewey, 1916(Dewey, , 1938) ) asserts that knowledge is actively constructed through student experiences (Dewey, 1938;Ertmer & Newby, 2013;Jaramillo, 1996).Likewise, during peer review, knowledge is constructed collaboratively through a shared learning experience with peers (Moneypenny et al., 2018).
In the Week Four and Seven scheduled peer review activities, each of the study participants took part in peer review conscientiously by posting their original work for review, reviewing their peers' work, and responding with feedback.Furthermore, in both weeks, every initial peer review response post provided a depth of more than 100 words.These findings support research conclusions by Dar et al. (2014) which claimed that when students are taught how and what to assess, the process can be simplified, and students' interest and motivation can be enhanced.
During the scheduled peer review activities, students assumed a responsible role through active engagement in first-hand, participatory learning.As emphasized in research by Clark (2018), during constructivism, a student is in control of his or her own learning.During the observation of the study participants' peer review engagement and the coding of the student post artifacts, it was evident to the researcher that students had utilized the structured peer evaluation system to prompt their active involvement.These findings support earlier research by Jaramillo (1996) which asserted that the constructivist learner is not a docile vessel waiting to receive knowledge but one who is strongly involved in the pursuit of his or her learning.

Structured System Prompted Social and Cognitive Liberation
During this research study, students demonstrated a strong degree of social expression and cognitive freedom via their peer review participation.

Social Presence
In support of Garrison and Arbaugh's (2007) established CoI categories and presence indicators, social presence was coded 349 times across the 24 student threads in the student post artifacts.Of the 598 coded occurrences of social and cognitive presence, social presence was more prevalent and coded 58% of the time.
The social presence of group cohesion was exhibited within the student post artifacts through agreements, compliments, and the use of encouraging conversation.Often, group cohesion included references to another student's work as the conversation became representative of several students working together to produce a resolution.Of the three categories of social presence coded within the student artifacts, group cohesion was coded for 210 of the 349 occurrences, representing 60% of all social presence.In the observational field notes, the researcher recorded a strong level of motivational support placed at the onset of the participants' peer review feedback.Coded as group cohesion, this initial delivery of affirmation and positivity aligned with the feedback sandwich example provided in the peer review tool kit:

Eastyn
I have thoroughly enjoyed watching your campaign unfold this term!I absolutely love the integrated strategy you've detailed in your report.
Furthermore, open communication through risk-free expression was coded a total of 123 times across the student post artifacts and took place in an open, uninhibited, and guilt-free manner: Marlo I struggled a bit with my organization of those three sections because you have so many ideas in your head it's hard to classify each one under the "right" section.
Lastly, affective expression was coded 16 times across the student post artifacts and was demonstrated through the use of emoticons in support of emotion, agreement, suggestion, and humor.These findings align with the social constructivist theory of learning as through dialogue, a collaborative culture of learning and student knowledge can be created and shaped through social interaction (Vygotsky, 1962).

Cognitive Presence
The design of the study's peer review tool kit was influenced by the constructivist theory of cognitive apprenticeship and its six dimensions (Brown & Stefaniak, 2016).During cognitive apprenticeship, implied processes are openly shared with students, as they visualize, participate in, and practice these processes with the instructor and their classmates (Collins et al., 1987).Based on Garrison and Arbaugh's (2007) established CoI categories and presence indicators, the student post artifacts were coded for 249 occurrences of cognitive presence.
The most highly coded cognitive presence was the category of exploration, offering 94 occurrences.Exploration was exhibited through suggestions to peers, brainstorming ideas, and the infusion of possible conclusions:

Skyler
And this may seem like a minor or silly thing or distinction to be making but I would consider not just targeting woman as your audience?
Although not as highly represented as the category of exploration, a triggering event was coded 69 times across the student post artifacts and was demonstrated through puzzlement or a sense of curiosity:

Salem
As I was reading your draft, I found that I was searching through the first few paragraphs trying to determine what type of school this campaign would be promoting.
Furthermore, the cognitive presences of integration and resolution were coded 41 and 45 times, respectively.Based on the established CoI categories and indicators (Garrison & Arbaugh, 2007), participants in this study presented a strong level of cognitive presence throughout peer review activities.Further sustaining these findings were the noted observations, as the researcher recorded a strong tendency for students to fully review the work of peers and deliver wellreasoned, well-researched responses.In addition, the researcher observed that students went above and beyond brief affirmative responses by providing links to outside resources, offering referrals back to prior instructor guidance, and citing and referencing valid sources to justify their claims.These findings support Boud's (2000Boud's ( , 2013) ) research assertions, which claimed that although peer review is utilized for assessment purposes, it fulfills an essential classroom component as students not only learn alongside each other but from one another as well.Following their engagement with the structured peer evaluation system, participants in this research study offered positive perceptions of the structured approach.Students reported an elevated degree of confidence and empowerment through their use of the peer review tool kit and openly acknowledged the collaborative community of learners that emerged.

Heightened Confidence and Empowerment Through Tool Kit Innovation
In feedback received through post-term questionnaire open-ended questions and one-onone interviews, students relayed an elevated level of self-confidence and empowerment due to the tool kit intervention:

Justice
In earlier peer review, there was no structure, but this gave you something to fall back on.It gave me more faith.

Eastyn
It's incredibly easy to feel underqualified, so I appreciated the reminders throughout the toolkit that showed me I was more than capable of helping my peers through a thoughtful review.
These findings support research that encourages the use of proactive training and support to help students understand how to give and receive peer review feedback prior to their participation (Alnasser, 2018;Baker, 2016;Dar et al., 2014;McMahon, 2010).
Furthermore, study participants acknowledged their peers' use of the tool kit: Salem I think they were a little more emboldened to give constructive criticism as opposed to platitudes.

Marlo
Yes! I could read between the lines when I received criticism that my peers had read guidelines to provide constructive criticism.
This feedback aligns with prior research by Barnard et al. (2015) which asserted that training can be provided to teach students how to deliver constructive feedback and to provide guidance for overly critical students (McMahon, 2010).Furthermore, study participants confirmed that specific peer review resources and tools helped to empower and support them:

Skyler
So, you have … what is a peer review and examples…I think that was helpful…It made me more knowledgeable.
Eastyn I really enjoyed the handout that had the diagram of the sandwich to remind us to preface the review with something positive, then offer constructive criticism, and then end on a high note.
These findings sustain research by Llado et al. (2014) which endorsed the use of unique approaches and training to clarify peer review tasks and deliver helpful tools and techniques.
Although this study offered a small number of study participants and the Cronbach's alpha score of the three parts of the preterm and post-term questionnaires offered low and varied consistency, there were some positive takeaways in support of students' perceptions with respect to peer review.In support of confidence in peers' ability to provide useful feedback, study participants provided a Likert scale response to the following statement in Part One of both questionnaires: The feedback my peers give me on my writing for this class will be useful.The mean score of the preterm questionnaire for Q5 (M=4.14) and the mean score of the post-term questionnaire for Q5 (M=4.43)offer positive implications.Following the intervention of the structured peer evaluation and the peer review tool kit innovation, students' perception of the usefulness of peers' feedback elevated slightly.

Collaborative Community of Learners Realized Through Peer Review Participation
Study participants perceived that their peer review interactions evolved into a collaborative community of learners who were invested in supporting one another.During the one-on-one interview, Oakley noted a peer review team approach and stated, "This week we're going to look at these things as a group and help each other get better." In response to the post-term questionnaire open-ended questions, Skyler stated, "Most explained their reasoning and thinking behind why they were making the suggestions they did…this made me more confident in accepting…what they had to say."Furthermore, in response to the one-on-one interview, Eastyn explained, "I know that through giving others peer review, it really did help me reflect on my own work and say…this is something that I should actually do in my project."By mirroring and practicing the skills they observe during peer review, students improve their work (Llado et al., 2014;Mulder et al., 2014).
During the one-on-one interview, Salem discussed the tool kit and the revelation that peers would be reviewing each other's work.Salem stated, "The section that talks about making me mindful of an initial draft, knowing someone is going to be reading it was probably my biggest takeaway…So, I feel like peer review helped me." Although interpretations of the preterm and post-term questionnaires should be tentatively considered, based on a limited number of students and low and varied internal consistency outcomes (DeVellis, 2016), an increase in the mean scores across relative questions from preterm to post-term was observed.In support of study participants' perception of increased interaction between peers during peer review activities, Likert scale responses were provided to this statement by students in Part Two of both the preterm and post-term questionnaires: Peer review activities increase the interaction between my classmates and me.The mean score of the preterm questionnaire for Q18 (M=4.57) and the mean score of the post-term questionnaire for Q18 (M=4.71)offer encouraging connotations.Following the intervention of the structured peer evaluation, the mean score for this statement elevated slightly, indicating the study participants' acknowledgment for the increased interaction that occurred during the Capstone term.Furthermore, students provided Likert scale responses to Q19, in Part Two of both the preterm and post-term questionnaires, which stated: Having a peer's feedback on a draft allows me to create a better final product.The mean score of the preterm questionnaire for Q19 (M=4.71) and the mean score of the post-term questionnaire for Q19 (M=4.86)produced a slight elevation from preterm to post-term.This slight growth denotes an increased appreciation for the collaborative feedback that study participants received across the community of learners within the Capstone classroom.Finally, one of the statements in Part Two of the preterm and post-term questionnaire was positioned to gauge students' feelings regarding the ability for peer review to foster community in an online learning environment.Study participants provided a Likert scale response to the following statement, entitled Q25: Peer review increases the sense of community in an online course.The mean score of the preterm questionnaire for Q25 (M=4.00) and the mean score of the post-term questionnaire for Q25 (M=4.57)produced an increase from preterm to post-term.This increase denotes the study participants' strong comprehension of the increase in community building that was experienced through the structured peer evaluation system.
Furthermore, in a review of the researcher's interview notations, a positive perception of peer engagement surfaced as a common theme.The researcher noted that Skyler shared a sense of enjoyment and proclaimed engagement to be the best part of peer review.Similar to research findings by Moneypenney et al. (2018), when students connect and share their understandings and experiences during peer review, a community of learners develops and grows.

Limitations, Implications, and Next Steps
The limitations of this research include a small sample size, lack of internal access to the externally located educational technology innovation, and potential researcher bias.However, this research offers implications and opportunities.Due to the study outcomes, a heightened expectation for student peer review participation should be realized, encouraged, and supported moving forward.The findings of this research study assert that students were empowered to move from hesitant bystander to one who was enthusiastically involved in a robust community of learners (Dar et al., 2014;Gikandi & Morrow, 2016;Kearney, 2013).Through a structured approach, scaffolded learning, and supportive tools and resources, students can obtain understanding and aptitude and become empowered to actively engage in peer review (Brown & Stefaniak, 2016).
As an additional implication, the vital role of learning theory in designing educational technology cannot be overstated.During this study, it was vital to design the tool kit innovation so that students of similar status could rise from independent problem-solving at the lower end of the zone of proximal development to a more advanced knowledge level and higher achievement (Vygotsky, 1978).By undergirding the tool kit innovation with theory, a learning pathway was created for students to construct knowledge through experience (Dewey, 1938;Ertmer & Newby, 2013;Jaramillo, 1996).
Recommendations for future research include the opportunity to place a tool kit intervention earlier in the learning pathway as an introductory training to teach students to collaborate, assess peers, and deliver proficient feedback (Sridharan et al., 2018).Furthermore, based on the outcomes of this study and the abundant existing literature, future researchers may consider the integration of additional resources to support students in overcoming peer review anxiety.

Declarations
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.Part Four: Open-ended Response Opportunities -6 Questions 1.What are your perceptions of the structured peer evaluation system that was provided to assist with peer review activities this term?
2. Did you access or use any of the resources or tools provided in the structured peer evaluation system in support of peer review activities?Why or why not?
3. Do you feel that the resources and tools in the structured peer revaluation system empowered you to offer serious and objective peer review feedback for your classmates?Why or why not?
4. Do you feel that the resources and tools in the structured peer evaluation system allowed you to feel more confident in accepting feedback received from your peers?Why or why not?
5. Do you feel that the use of the resources and tools in the structured peer evaluation system promoted a sense of community among peers during peer review activities?Why or why not?
6. What other comments would you like to add about the structured peer evaluation system?SA A N D SD Field Notes RQ2: What are the perceptions of students regarding a structured peer evaluation system in support of online asynchronous peer review activity in a Graduate Communication Capstone classroom at UNCM? • Preterm and Post-term Questionnaires (Parts One, Two, and Three) • One-on-One Interviews • Researcher's Handwritten Interview Notations • Post-term Questionnaire (Part Four) Question 2: What are the Perceptions of Students Regarding a Structured Peer Evaluation System in Support of Online Asynchronous Peer Review Activity in a Graduate Communication Capstone classroom at UNCM?
will give me a fair grade on my writing.SA A N D SD 10.I will receive a fair assessment of my work through the peer review given to me by multiple peers.review is helpful to my learning.(POS) SA A N D SD 12. Peer review makes me better understand an assignment's requirements.(POS) SA A N D SD 13.Peer review activities can improve my skills in verbal communication.(POS) SA A N D SD 14. Peer review activities can improve my skills in written communication.(POS) SA A N D SD 15.Peer review activities motivate me to learn.(POS) SA A N D SD 16.Peer review activities increase the interaction between my teacher and me.(POS) SA A N D SD 17. Peer review helps me develop a sense of participation in a course.(POS) SA A N D SD 18. Peer review activities increase the interaction between my classmates and me.(POS) SA A N D SD 19.Having a peer's feedback on a draft allows me to create a better final product.(POS) SA A N D SD 20.Receiving feedback from my peers can be just as valuable as receiving feedback from my professor.(POS) SA A N D SD 21.Submitting a project to my peers can be intimidating.(NAS) SA A N D SD 22.I think students should not be responsible for making assessments.(NAS) SA A N D SD 23.Peer review is time-consuming.(NAS) SA A N D SD 24.My comments given to other classmates are affected by comments given to me.(NAS) SA A N D SD 25.Peer review increases the sense of community in an online course.(OAS) SA A N D SD 26.Online peer review activities can be time-saving.(OAS) SA A N D SD 27.Online course peer review can be as effective as face-toface course peer review.(OAS) Part Three: Understanding and Action -3 Questions SA A N D SD 28.Peer review activities help me understand what other classmates think.(UAS) SA A N D SD 29.The teacher should develop criteria (such as a rubric or guide) for students completing peer review.(UAS) SA A N D SD 30.Students should participate in the development of criteria (such as a guide or a rubric) for peer review.(UAS) SA A N D SD Fairness of peers' feedback 9. Peers gave me a fair grade on my writing.SA A N D SD 10.I received a fair assessment of my work through the peer review given to me by multiple peers.review activities can improve my skills in written communication.(POS) SA A N D SD 15.Peer review activities motivate me to learn.(POS) SA A N D SD 16.Peer review activities increase the interaction between my teacher and me.(POS) SA A N D SD 17. Peer review helps me develop a sense of participation in a course.(POS) SA A N D SD 18. Peer review activities increase the interaction between my classmates and me.(POS) SA A N D SD 19.Having a peer's feedback on a draft allows me to create a better final product.(POS) SA A N D SD 20.Receiving feedback from my peers can be just as valuable as receiving feedback from my professor.(POS) SA A N D SD 21.Submitting a project to my peers can be intimidating.(NAS) SA A N D SD 22.I think students should not be responsible for making assessments.review is time-consuming.(NAS) SA A N D SD 24.My comments given to other classmates are affected by comments given to me.(NAS) SA A N D SD 25.Peer review increases the sense of community in an online course.(OAS) SA A N D SD 26.Online peer review activities can be time-saving.(OAS) SA A N D SD 27.Online course peer review can be as effective as face-to-face course peer review.(OAS) teacher should develop criteria (such as a rubric or guide) for students completing peer review.(UAS) SA A N D SD 30.Students should participate in the development of criteria (such as a guide or a rubric) for peer review.(UAS)

Table 4
Community of Inquiry Findings to Themes with Examples and Prior Research The author/researcher declared completion of the Collaborative Institutional Training Initiative (CITI Program) for Human Research (Social & Behavioral Researchers) and the receipt of UNCM IRB approval prior to conducting the research study.If my peers gave me feedback several months from now on the same work, they will examine for this class, they would probably give me similar feedback.