Adapting the Communication and Teamwork Skills Assessment to Assess Pre-licensure Health Care Student Team Performance in Simulation-Enhanced Interprofessional Education

INTRODUCTION Literature demonstrates a lack of structured reporting of interprofessional (IPE) education activities and challenges objectively measuring team performance. The purposes of this article are: to provide a structured description of a simulation-enhanced IPE project focusing on pre-licensure health care student team performance; and, to describe how the Communication and Teamwork Skills (CATS) assessment was adapted to assess teamwork and communication skills during student team simulations. METHODS Simulated case scenarios were conducted with teams consisting of nursing, respiratory therapy, and medical students. The CATS tool was adapted for use in IPE simulations by adding two statements that globally assessed frequency and quality of teamwork. Individual items from the CATS tool guided assessment of overall team performance. Faculty assessors piloted the tool by discussing tool components and assessing several sample cases together. Faculty assessors then scored each simulation individually and the adapted CATS tool was assessed for interrater reliability. RESULTS The team assessed sixty-eight cases (n=68). Thirty-four (n=34) cases were rated by three of the faculty and thirty-four (n=34) were rated by two faculty. Inter-rater reliability for frequency of communication was .808 and .789 for quality of communication. CONCLUSION This project suggests that an adapted version of the CATS assessment tool can be used to reliably assess communication performance of health care student teams during a simulated acute care case. In addition, the planning team hopes that the project can be replicated to develop a model of IPE that is sustainable and feasible within other academic or health care settings. Received: 03/31/2017 Accepted: 04/02/2018 © 2018 Masters, et al.. This open access article is distributed under a Creative Commons Attribution License, which allows unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. H IP & Adapting the Communication ORIGINAL RESEARCH 3(3):eP1159 | 2 Introduction Educators are challenged to infuse core competencies related to teamwork and communication into curricula to ensure pre-licensure health care students are prepared to function in interdisciplinary teams (Institute of Medicine [IOM], 2003; IOM, 2010; Interprofessional Education Collaborative Panel [IPEC], 2016). A key concept associated with this work is interprofessional education (IPE). According to The World Health Organization (2010), IPE occurs “when students from two or more professions learn about, from, and with each other to enable effective collaboration and improve health outcomes” (p. 13). One strategy to teach IPE competencies is simulation, which provides a controlled and safe environment for students to practice their teamwork and communication skills (Decker et al., 2015; National League for Nursing [NLN], n. d.). Integration of IPE into pre-licensure health care education has gained momentum over the past several years in an effort to improve patient care outcomes. Students can work towards this goal through development of mutual respect and improved understanding of other disciplines’ expertise (IPEC, 2016). A large-scale literature review focusing on pre-licensure health care IPE identified 83 studies that had been published between 2005 and 2010 (Abu-Rish et al., 2012). The Interprofessional Education Collaboration was founded in 2009 to “prepare future health professionals for enhanced team-based care of patients and improved population health outcomes” (IPEC, 2016, p. 1). This organization supports health care educators through establishment of core IPE competencies, IPEC Faculty Development Institutes, and a newly created IPEC-PORTAL with free, high-quality teaching materials. Today, membership includes professional education institutions representing fifteen diverse health care professions. Accreditation agencies and healthcare organizations such as the NLN, Society for Simulation in Health Care, American Association of Colleges of Nursing, International Nursing Association for Clinical Simulation and Learning, and Institute for Healthcare Improvement have endorsed Sim-IPE as an effective mechanism to teach IPE (Decker et al., 2015; NLN, n. d.). Sim-IPE takes place when “participants and facilitators from two or more professions are engaged in a simulated health care experience to achieve shared or linked objectives and outcomes” (Decker et al., 2015, p. 294.). There has been an increase in published Sim-IPE studies over the past 10 years, with reports of positive outcomes for students elicited through both informal and formal evaluation. Palaganas, Brunette, and Winslow (2016) suggests that students enjoyed the realism, practice, debriefing, reflection, relevancy, and opportunities for feedback associated with Sim-IPE. Students also perceived improvement in their knowledge, skills, and behaviors related to teamwork, communication, and collaboration. Pre-licensure IPE and Sim-IPE activity has increased, but concerns have been identified with the existing work. One weakness reported in the literature is a perceived lack of rigor in outcome assessment including limited psychometric testing and inadequate psychometric development of existing tools (Abu-Rish et al., 2012; Decker et al., 2015; Palaganas et al., 2016). Recent literature reviews found that the majority of IPE Implications for Interprofessional Practice • An adapted Communication and Teamwork Skills (CATS) assessment tool demonstrated initial reliability in assessing pre-licensure health care student team performance in simulation-enhanced interprofessional education (Sim-IPE). • Dissemination of interprofessional education (IPE) utilizing a standardized format may foster replication of similar projects in academic and health care settings. • Long-term commitment is required by educators to facilitate high-quality, sustainable IPE experiences for students in health care programs. H IP & ISSN 2159-1253 Health & Interprofessional Practice | commons.pacificu.edu/hip 3(3):eP1159 | 3 and Sim-IPE studies did not address reliability and validity of measurement tools (Abu-Rish et al., 2012; Palaganas et al., 2016). Another criticism of current efforts to evaluate IPE is limited measurement of team performance skills, with the majority of tools and studies focused on assessing participants’ knowledge, attitudes, and perceptions about IPE concepts and practice (National Center for Interprofessional Practice and Education, n.d.). This finding is consistent with the literature in that only 9.6% of IPE studies developed and implemented observational ratings for assessing students’ communication and teamwork skills (Abu-Rish et al., 2012). Tools that measure team performance are available (Agency for Healthcare Research and Quality [AHRQ], n.d.; Chiu et al., n.d.; Frankel, Gardner, Maynard, & Kelly, 2007; Kiesewetter & Fischer, 2015; Malec et al., 2007). However, it is difficult to identify an appropriate and reliable tool to use in pre-licensure IPE that quantitatively measures communication and teamwork skills and is simple to use. A second concern associated with IPE projects is inconsistency and lack of description when disseminating findings. Standardized methods of reporting, including focus on IPE preparation, activities, and outcomes, was strongly encouraged to address these shortcomings (Abu-Rish et al., 2012; Olson & Bialocerkowski, 2014; Palaganas et al., 2016). Suggestions for specific variables to be included when reporting are: theoretical or conceptual framework, intervention design, simulation scenarios and modality, demographics of students and faculty, teaching strategies, frequency and duration of IPE intervention, faculty development, validation of measurement tools, institutional leadership support, barriers and facilitators, and community partnerships (Abu-Rish et al., 2012; Olson & Bialocerkowski, 2014; Palaganas et al., 2016). An example of a standardized method for reporting IPE, entitled the Replicability of Interprofessional Education (RIPE) tool, can be viewed in Table 1. The objective of standardized reporting is to foster replication and encourage comparison of IPE initiatives with the ultimate goal of improving rigor of IPE and Sim-IPE research. Integration of IPE into pre-licensure health care curricula is essential to prepare students for the realities of practice. However, IPE research must meet rigorous scientific standards and be replicable in academia and practice. Thus, the goals of this article are: To utilize RIPE guidelines to describe a pre-licensure Sim-IPE project between a community hospital and private college focusing on team performance To describe adaptation of the CATS assessment for use as an assessment tool for pre-licensure health care student team performance


Introduction
Educators are challenged to infuse core competencies related to teamwork and communication into curricula to ensure pre-licensure health care students are prepared to function in interdisciplinary teams (Institute of Medicine [IOM], 2003;IOM, 2010;Interprofessional Education Collaborative Panel [IPEC], 2016). A key concept associated with this work is interprofessional education (IPE). According to The World Health Organization (2010), IPE occurs "when students from two or more professions learn about, from, and with each other to enable effective collaboration and improve health outcomes" (p. 13). One strategy to teach IPE competencies is simulation, which provides a controlled and safe environment for students to practice their teamwork and communication skills (Decker et al., 2015; National League for Nursing [NLN], n. d.).
Integration of IPE into pre-licensure health care education has gained momentum over the past several years in an effort to improve patient care outcomes. Students can work towards this goal through development of mutual respect and improved understanding of other disciplines' expertise (IPEC, 2016). A large-scale literature review focusing on pre-licensure health care IPE identified 83 studies that had been published between 2005(Abu-Rish et al., 2012. The Interprofessional Education Collaboration was founded in 2009 to "prepare future health professionals for enhanced team-based care of patients and improved population health outcomes" (IPEC, 2016, p. 1). This organization supports health care educators through establishment of core IPE competencies, IPEC Faculty Development Institutes, and a newly created IPEC-PORTAL with free, high-quality teaching materials. Today, membership includes professional education institutions representing fifteen diverse health care professions.
Accreditation agencies and healthcare organizations such as the NLN, Society for Simulation in Health Care, American Association of Colleges of Nursing, International Nursing Association for Clinical Simulation and Learning, and Institute for Healthcare Improvement have endorsed Sim-IPE as an effective mechanism to teach IPE (Decker et al., 2015;NLN, n. d.). Sim-IPE takes place when "participants and facilitators from two or more professions are engaged in a simulated health care experience to achieve shared or linked objectives and outcomes" (Decker et al., 2015, p. 294.). There has been an increase in published Sim-IPE studies over the past 10 years, with reports of positive outcomes for students elicited through both informal and formal evaluation. Palaganas, Brunette, and Winslow (2016) suggests that students enjoyed the realism, practice, debriefing, reflection, relevancy, and opportunities for feedback associated with Sim-IPE. Students also perceived improvement in their knowledge, skills, and behaviors related to teamwork, communication, and collaboration.
Pre-licensure IPE and Sim-IPE activity has increased, but concerns have been identified with the existing work. One weakness reported in the literature is a perceived lack of rigor in outcome assessment including limited psychometric testing and inadequate psychometric development of existing tools (Abu-Rish et al., 2012;Decker et al., 2015;Palaganas et al., 2016). Recent literature reviews found that the majority of IPE

Implications for Interprofessional Practice
• An adapted Communication and Teamwork Skills (CATS) assessment tool demonstrated initial reliability in assessing pre-licensure health care student team performance in simulation-enhanced interprofessional education (Sim-IPE).
• Dissemination of interprofessional education (IPE) utilizing a standardized format may foster replication of similar projects in academic and health care settings.
• Long-term commitment is required by educators to facilitate high-quality, sustainable IPE experiences for students in health care programs. and Sim-IPE studies did not address reliability and validity of measurement tools (Abu-Rish et al., 2012;Palaganas et al., 2016). Another criticism of current efforts to evaluate IPE is limited measurement of team performance skills, with the majority of tools and studies focused on assessing participants' knowledge, attitudes, and perceptions about IPE concepts and practice (National Center for Interprofessional Practice and Education, n.d.). This finding is consistent with the literature in that only 9.6% of IPE studies developed and implemented observational ratings for assessing students' communication and teamwork skills (Abu-Rish et al., 2012). Tools that measure team performance are available (Agency for Healthcare Research and Quality [AHRQ], n.d.; Chiu et al., n.d.;Frankel, Gardner, Maynard, & Kelly, 2007;Kiesewetter & Fischer, 2015;Malec et al., 2007). However, it is difficult to identify an appropriate and reliable tool to use in pre-licensure IPE that quantitatively measures communication and teamwork skills and is simple to use.
A second concern associated with IPE projects is inconsistency and lack of description when disseminating findings. Standardized methods of reporting, including focus on IPE preparation, activities, and outcomes, was strongly encouraged to address these shortcomings (Abu-Rish et al., 2012;Olson & Bialocerkowski, 2014;Palaganas et al., 2016). Suggestions for specific variables to be included when reporting are: theoretical or conceptual framework, intervention design, simulation scenarios and modality, demographics of students and faculty, teaching strategies, frequency and duration of IPE intervention, faculty development, validation of measurement tools, institutional leadership support, barriers and facilitators, and community partnerships (Abu-Rish et al., 2012;Olson & Bialocerkowski, 2014;Palaganas et al., 2016). An example of a standardized method for reporting IPE, entitled the Replicability of Interprofessional Education (RIPE) tool, can be viewed in Table 1. The objective of standardized reporting is to foster replication and encourage comparison of IPE initiatives with the ultimate goal of improving rigor of IPE and Sim-IPE research.
Integration of IPE into pre-licensure health care curricula is essential to prepare students for the realities of practice. However, IPE research must meet rigorous scientific standards and be replicable in academia and practice. Thus, the goals of this article are: To utilize RIPE guidelines to describe a pre-licensure Sim-IPE project between a community hospital and private college focusing on team performance To describe adaptation of the CATS assessment for use as an assessment tool for pre-licensure health care student team performance

Participants
A partnership between a large community hospital and small private college in close proximity to the hospital was established to facilitate a Sim-IPE project. The hospital is a 580-bed community teaching hospital and Level 1 trauma center with seven residency programs, four major medical school affiliations, and five allied health schools. The college is a coeducational four-year institution offering fifty undergraduate majors including nursing and respiratory therapy. For this project, the college assigned nursing and respiratory care students to the simulations because they were on clinical rotations at the same hospital as the medical students. Nursing students were in their junior level medical-surgical rotations. Respiratory therapy students were most often in their final semester of study. A variety of colleges and universities provided third or fourth year medical students while on clinical rotations at the hospital. Registered nurses on the hospital's Rapid Response Team (RRT) also played a vital role mentoring students during the simulation experiences. They were available as a resource for student teams, providing guidance in assessment and interventions and encouraging informal debriefing. The diversity of participants mimicked the realities of clinical practice, necessitating interaction between individuals with different levels of knowledge, experience, and expertise to problem solve in a simulated clinical environment.
Faculty from both institutions were involved as intervention developers. Hospital faculty included the physician-director of medical education, a clinical nurse specialist, and the director of the simulation laboratory. Two nursing professors and the director of clinical simulation represented the college. The Sim-IPE was also supported by faculty of the clinical education-respiratory care program. All faculty willingly volunteered to participate in this project during regularly scheduled work hours. No additional compensation was provided to faculty and no additional funding was provided for project implementation. While some faculty had a ba-

Implementation
Planning began the summer before implementation when several meetings were held to develop an appropriate case study, discuss scheduling of students, and to consider possible outcome evaluation methods. Institutional Review Board (IRB) approval was addressed with both institutions. The research department at the hospital did not require the project to go through the IRB process. The college's IRB approved the project through an expedited review. Based on the composition of student participants, the planning team decided that an acute care respiratory case study was most appropriate. This would require input from each discipline as the scenario progressed. The group also decided not to provide a detailed patient history to the nursing students prior to initiation of the simulation so students would need to speak directly to the patient and one another to elicit relevant information. In order to force the need to seek assistance from additional resources, the scenario was designed such that in spite of students' actions, the "patient" did not improve. The planning group followed a prebriefing script and debriefing guidelines written by the hospital simulation director.
The primary goal of this Sim-IPE project was to assess teamwork and communication skills of pre-licensure student teams, not medical or clinical knowledge of individual participants. The major challenge of this project was identification of an appropriate tool to measure these skills. Tools considered for use were the: The team chose to further evaluate the CATS tool for use because it demonstrated reliability, did not necessitate identification of a team leader, and utilized Team-STEPPS as a framework (AHRQ, n.d.). Team members were familiar with TeamSTEPPS and preferred use of similar terminology in the selected tool. A perceived negative aspect of the CATS tool was complexity in the scoring process. This included the requirement to mark each time a team behavior occurred and to grade its quality. Team members felt that this would be a challenge, particularly as related to time and resources required for assessors to be trained in tool usage. An author of the CATS tool was contacted to discuss her experiences using the tool and elicit input regarding planning team concerns. Permission was granted for the tool to be adapted for use with pre-licensure health care teams. Four categories from the original CATS were maintained on the adapted tool: communication, situational awareness, cooperation, and coordination. A "cheat sheet" of performance behaviors associated with each category was added to the back of the tool as a reference. The team added two global questions related to frequency and quality of communication/teamwork and a comments section. By adding the two questions and the comment section, assessors were able to rate team performance quantitatively and provide feedback qualitatively during debriefing. The adapted tool can be viewed in Table 2.

Description of IPE Intervention
Simulation days were held at the hospital's simulation center six to eight times during the semester, depending on the number of nursing clinical groups. Three to four simulations were conducted on each simulation day, with the majority of students participating in one 30 minute simulation followed by a 30 minute debriefing. Several respiratory students participated in more than one simulation because there were fewer students in that program. Participation was required by all health care students as a portion of their clinical experiences. Students were not graded on their performance during the simulations.
Early in the semester, students were informed that they would be participating in the IPE activity. At their assigned times, faculty and student participants reported to the center where introductions were done and a generalized overview of IPE was provided. Student groups received a standardized orientation to the simulation room by the hospital's simulation lab director. This prebriefing included an overview of use of the mannequin, location of supplies, utilization of monitoring equipment, how to "order" diagnostic and lab values, and

*SBAR: Situation -Background -Assessment -Recommendation
Global Observation -frequency of communication/teamwork behaviors  Briefing A conversation and 2-way dialogue of concise and relevant information shared prior to a procedure or activity.
Verbalize plan Speak aloud the next steps for the procedure and/or care of the patient Verbalize expected time frames Speak aloud time frames for particular interventions. "We'll give this another 2 minutes and if there's no change we'll try X" Debriefing A conversation and 2-way dialogue of concise and relevant information shared after the procedure or activity is completed.

Situational awareness
Visually scan environment Clinicians look up, look at one another, look at equipment, and look around the room.

Verbalize adjustments in plan as changes occur
Speak aloud new plans, changes in strategy or intervention, and new time lines as procedure progresses Cooperation Request external resources if needed Speak aloud, asking for help from outside the team-other clinicians, rooms, equipment, consults, etc. Ask for help from team as needed Team members speak aloud, asking for assistance from members of the team.

Verbally request team input
Ask aloud for team's suggestions, opinions, comments, or ideas

Cross monitoring
Acknowledge concerns of others-watching team members, awareness of their actions, verbally stating concerns, sharing work load, verbally updating others in manner less formal than briefing, responding to concerns of team members Verbal assertion "Speak Up" If team members are uncomfortable or unclear, they speak aloud their concerns and state an alternative viewpoint or suggest an alternative course of action.

Receptive to assertion and ideas
Team members open to one another's concerns and suggestions

Communication
Closed loop When a request is made of team members, someone specifically affirms aloud that they will complete the task and states aloud when the task has been completed SBAR (situationbackground-assessmentrecommendation) on) Use of specific structured communication that states the situation, background, assessment, and recommendation.

Verbal updates-think aloud
Think aloud-Team members verbally state their perceptions, actions, and plans as the procedure progresses Use names Use team members' names Communicate with patient Team members speak to and respond to patient Appropriate tone of voice Team members use a tone of voice that is calm, professional, and not unnecessarily loud how to access additional resources. Student teams were then formed, and the process began. Students waiting for their turn to do the scenario were encouraged to interact with one another to help build team rapport and alleviate anxiety about their performance during the IPE activity.
Two nursing, one respiratory therapy, and one medical student participated in each scenario. The students had no knowledge of the scenario prior to entering the room. They were instructed to treat the simulation as an actual patient situation. Two nursing students received a brief history and were the first to enter the room, answering the call light and interacting with the "patient. " The expectation was for the students to perform physical assessments, gather relevant diagnostic data, and administer appropriate treatments and medications if ordered. As the case evolved, students would call for appropriate assistance from other team members. The student team was also instructed that higher level resources were available as would be the case in real life. Phone numbers for attending physicians/consultants, pharmacy, and the RRT were available next to the phone in the room. There was a whiteboard that could be utilized as needed. The medical student and the nursing students also were required to "give report" to their appropriate team member in the receiving unit prior to transferring the patient. As students entered the room, the assessors would rate team dynamics using the adapted CATS assessment.

Results
During the first semester of project implementation, faculty assessors who were also members of the planning team observed simulations together. Simulations were monitored via a one-way mirror in an observation room separate from the simulation room. This enabled assessors to carefully observe behaviors and come to a consensus as to which teamwork and communication behaviors occurred. Assessment scores were compared which facilitated discussion as to why student teams received the associated frequency and quality scores. These were essential steps within the process, ensuring that assessors had a shared mental model when formally assessing student team performance to determine reliability of the adapted CATS tool.
After the tool was piloted, the team assessed sixty-eight cases (n=68) over the course of the next two semesters. Each faculty assessor rated the scenario independently, sitting in different areas in the observation room and not discussing their impressions or assessments of student teams. Thirty-four (n=34) cases were rated by three faculty and thirty-four (n=34) were rated by two faculty. Intra-class correlation was used to assess interrater reliability. The inter-rater reliability (IRR) for frequency of communication was .808 and the IRR for quality of communication was .789. SPSS (IBM Corp. Released 2017. IBM SPSS Statistics for Windows, Version 23.0. Armonk, NY: IBM Corp.) was used to analyze data.
Debriefing provided an informal method of outcome assessment. After each simulation, student teams participated in a debriefing session facilitated by the director of simulation from the college or hospital. To encourage consistency, the simulation directors used the previously-mentioned debriefing guidelines and a template they had developed collaboratively. Each small group was taken out of the simulation area to another room where the debriefing was held. The debriefing encouraged students to reflect on the simulation, considering their team experience for coordination, situational awareness, cooperation, and collaboration (the four categories of the adapted CATS tool). The goal of debriefing was for each student to develop a take-home lesson related to teamwork skills. Examples included: "Everyone has a voice, " "Each newcomer gets report, " "Use closed loop communication, " "Call for help, " and "Keep patient in the loop. " After simulations and debriefings were completed, all participating students and Sim-IPE faculty came together for a large debriefing session. The student contributors were asked about the positive aspects of the experience as well as opportunities for improvement. The students commented on their own simulation experience and were not critical of other teams they may have observed. Respondents thought the exercise was "eye-opening" and "helpful. " Though some expressed a familiarity with IPE, all stated that they had not participated in sessions such as this one. Some participants stated, "It was really great getting to be with the other members of the team" or "After this, I really feel like it is an actual team caring for the patient. I am not alone. " experience for three pre-licensure health care student disciplines and the development of an adapted CATS tool to evaluate teamwork skills. The first major support was that our team of educators was committed to participating in this project over a long period of time and consisted of at least one member from nursing, respiratory, and medicine. Representation from all involved disciplines is essential for sustainability and promotion of learning for all students (Palaganas et al., 2016). Many IPE cases only involve nursing and medicine; participation by at least one other discipline is more realistic to actual practice.

Supports of Intervention
Our second major support was the level of expertise of our team of educators. Simulation experts ran the simulation cases and conducted the debriefing sessions. This included an experienced simulation "actor" as the voice of the mannequin who was familiar with the case study and could appropriately answer student questions to enhance simulation flow. Having experts in debriefing was critical to facilitating reflection, which enhanced student learning. Experienced clinicians, including faculty and members of the RRT, served as consultants for student teams which provided a realistic view of clinical practice.
The third support was the tool selected to assess team performance. The adapted CATS tool was reliable and relatively easy to use with trained observers, because it did not involve identifying and evaluating each individual behavior. It is important to establish rater consistency (Kardong-Edgren, Oermann, Rizzolo, & Odom-Maryon, 2017). Our raters exhibited consistency in professional judgment in assessing simulation cases on multiple days over a period of time. The adapted tool provided a simple method of assessing team communication skills. During simulations, faculty gathered data using the specific behaviors on the CATS tool. These data were used in debriefing to provide feedback to the students in an effort to highlight desired team behaviors for future practice.
Although we did not formally assess student attitudes towards the Sim-IPE, the students were generally positive and enthusiastic about their experience. This has been a consistent finding in multiple studies on Sim-IPE experiences (Palaganas et al., 2016). Another positive outcome of this experience was the opportunity for students from different disciplines to spend time with each other while they were waiting for their turn in the simulation lab. Many students verbalized that these interactions allowed them to learn more about other educational programs and helped them to gain understanding of other disciplinary roles.

Barriers to Intervention
Similar to other IPE reports, scheduling the simulations had some challenges (Palaganas et al., 2016). We attempted to address this issue by scheduling the dates a few months in advance. However, the medical students sometimes did not receive much advance notice regarding the simulation experience due to the nature of their four-week rotations. This resulted in occasional confusion in the schedule.
As previously discussed, a second challenge was selecting a tool that included the communication behaviors we hoped to observe in an interdisciplinary team and did not emphasize the role of a team leader. We did not want to have an assigned team leader in an effort to demonstrate that any member of the team can take the lead depending on the situation. Because none of the planning team members had formalized IPE training, we spent extensive time reviewing the literature and engaging in self-teaching during the initial project planning phase. This approach worked for our team, but participation in a formalized team training program may be valuable and decrease the initial planning time.
The significant time commitment required to develop, plan, and implement Sim-IPE was a third challenge. While we did not encounter any administrative barriers in respect to educator or student participation, the project was conducted in addition to regular work responsibilities and without funding. In addition to the planning time, it was also time-consuming for the raters to develop a shared mental model when utilizing the adapted CATS tool. Shared mental models, as described by McComb and Simpson (2014), are "individually held knowledge structures that help individual team members to function collaboratively in their environments and are comprised of four elements: content, similarity, accuracy and dynamics" (p. 1485). While this model is most often applied to health care teams, it is also applicable in tool development as raters strive to utilize specific content to develop similarity and accuracy in the evaluation process. Another challenge the raters had to overcome was to focus on assessing team performance and not individual performance.
Because the raters were all nursing faculty, they had to ORIGINAL RESEARCH 3(3):eP1159 | 10 make an intentional effort to focus on rating the overall team performance and not rate based on the nursing students' performance, particularly if they performed below expectations.

Lessons Learned
Despite the barriers to implementing Sim-IPE for prelicensure health care students, our positive experiences and convictions regarding the value of this work outweighed our challenges. The long-term commitment required by educators to facilitate IPE experiences for students in health care programs cannot be underestimated. Educators interested in developing IPE experiences that create realistic situations for students to practice their communication skills should advocate for administrative support in the form of time, compensation, and/or professional recognition. Having participation and voice from all disciplines and project participants in creating a learning experience is essential for an IPE experience to be successfully implemented and sustained. Additionally, members of our planning team committed to focusing on team training and not on skills training, which has been found to be a barrier in Sim-IPE (Palaganas et al., 2016). Educators must also work to role model effective team behaviors with each other and with students during pre-briefing, simulation, and debriefing.
Development of the adapted CATS tool is a first step in creating a simple method to assess Sim-IPE team performance with pre-licensure healthcare students. Further validation of this modified tool is necessary to confirm our reliability findings. Based on our Sim-IPE observations, we recognized the need to include more training in teamwork skills during educational preparation. Very few student groups demonstrated exceptional teamwork and communication skills. We found that students were generally weak in their ability to give a concise Situation-Background-Assessment-Recommendation (SBAR) report. Most student groups did not verbalize the plan to the patient, debrief after an intervention, or demonstrate closed loop communication. In response, the planning team has developed a brief presentation addressing teamwork and communication skills to share with students prior to future Sim-IPE activities.
Students in health care disciplines would benefit from several IPE experiences, because it is unrealistic to expect students to demonstrate strong communication skills with other health care providers without multiple opportunities to practice these skills. Current Sim-IPE literature is unclear regarding student exposure to each simulation (Palaganas et al., 2016). Additional research is needed to determine how duration of a simulation affects student learning. Many of the students verbalized they would like another opportunity to participate in Sim-IPE to integrate some of the feedback they received in debriefing. Therefore, longitudinal studies to assess students' communication skills at a later point in their education are also warranted.

Conclusion
This project demonstrates a sustainable partnership between a large community hospital and a small private college that provided an opportunity for students from three disciplines to gain an understanding of each other's roles on the health care team. Strengths of this project include development of a simple, reliable tool to assess teamwork performance, and potential for project replicability. Without simple, reliable, and valid evaluative tools, it will be challenging to measure student's teamwork and communication performance. In addition, we strongly hope utilization of the RIPE tool will provide standardized guidance to promote replication of similar Sim-IPE projects in both academic and health care settings. The ability to demonstrate strong teamwork and communication skills is an essential competency which enables health care professionals to provide safe, high-quality health care (IOM, 2003;IOM, 2010;IPEC, 2016.) According to Reeves, Perrier, Goldman, Freeth, and Zwarenstein (2013), while pre-licensure IPE cannot currently be associated with patient outcomes or processes, it can be regarded as an "investment in the future. " Creating innovative opportunities for pre-licensure health care students to practice teamwork and communication skills is an important step toward linking IPE to improved health outcomes.