Preceptor Teaching Behavior and the Previous Objective Structured Teaching Examination (OSTE): A Correlational Study

Introduction: It is important for nurse educators to evaluate the teaching performance of nurse preceptors related to these workplace-based assessment tools. The objective structured teaching evaluation was developed to evaluate teaching performance. However, limited attention has been given to the relationship between the use of these workplace-based assessment tools and previous OSTE training. Hence, this study is to investigate the relationship between the teaching behaviors of nurse preceptors and previous objective structured teaching evaluation (OSTE) training. Methods: This cross-sectional study recruited 42 N3-level nurse preceptors who had completed previous OSTE training at a medical teaching hospital in central Taiwan. Demographics, teaching behaviors, and OSTE-related variables of the participants were assessed using a questionnaire. Results: For assessment tools, 90.6% of the nurse preceptors used direct observations of procedural skills (DOPS), 59.4% used mini-clinical evaluation exercises (mini-CEX), and 40.6% used case-based discussions (CBD). The sandwich feedback technique was used for clinical teaching evaluations. Shown out of the passing score, the mean scores of the nurse preceptors for each OSTE station were 82.9/66.8 for DOPS, 71.0/73.3 for mini-CEX, and 69.4/71.5 points for CBD. The use of all three assessment tools was correlated with use of the sandwich feedback technique (r = 0.94–1, p < 0.01). The OSTE scores were not correlated with teaching behaviors (r = −0.06–0.25, p > 0.01). Conclusions: Nurse preceptors exhibited good teaching behaviors, using workplace-based assessment tools and the sandwich feedback technique. Clinical teaching behaviors may be more dependent on hospital teaching policies than Lee Y, Chen C, Kao Y, Yang M, Wu S, Wen Y MedEdPublish https://doi.org/10.15694/mep.2020.000139.1 Page | 2 on previous OSTE training scores.


Introduction
Nurse preceptors play a significant role in clinical education (Condrey, 2015;Shinners and Franqueiro, 2015). To be effective in their teaching role, nurse preceptors need to possess knowledge and practical skills, as well as training skills. This involves knowing how to implement curriculum designs, teaching assessment skills, and feedback techniques (Condrey, 2015;Joint Commission of Taiwan, 2018;Lee et al., 2017;Shinners and Franqueiro, 2015). In Taiwan, nurse preceptors often use workplace-based assessment tools to assess the performance of novice nurses in clinical teaching. These workplace-based assessment tools used are often the direct observation of procedural skills (DOPS), mini-clinical evaluation exercises (mini-CEX), and case-based discussions (CBD) (Joint Commission of Taiwan, 2018; Swanwick and Chana, 2009). The DOPS was originally developed by the Royal College of Physicians to assess competencies in technical skills, procedures, and protocols (Norcini and McKinley, 2007). The mini-CEX was originally developed by the American Board of Internal Medicine to evaluate the competencies of residents in communication with patients, physical examination, diagnosis, and treatment planning (American Board of Internal Medicine, 2020). The CBD was adapted from the "chart-stimulated recall" used in North America for the education of medical and allied healthcare professionals to assess competencies in clinical judgement, clinical management, and reflective practice (Awad et al., 2014).
It is important for nurse educators to evaluate the teaching performance of nurse preceptors related to these workplace-based assessment tools. The objective structured teaching evaluation (OSTE) was developed to evaluate teaching performance (Julian et al., 2012;Sturpe and Schaivone, 2014). However, limited attention has been given to the relationship between the use of these workplace-based assessment tools and previous OSTE training. To address this gap in the literature, this study investigated the relationship between the teaching behaviors of nurse preceptors and previous OSTE training at a medical teaching hospital in Taiwan.
The OSTE was designed to evaluate teaching performance in the clinical environment (De Grasset et al., 2018). The OSTE consists of faculty training activities in which teachers (as learners) perform teaching activities with a standardized student (Sturpe and Schaivone, 2014). Each OSTE encounter is the same; a standardized patient or other stimulus is represented, and the examinee interacts with the standardized student, who has been trained to respond in a consistent manner (Julian et al., 2012). The examinee receives immediate feedback (Julian et al., 2012). The OSTE is often used to assess teaching behavior in medical education and other disciplines (De Grasset et al., 2018;Julian et al., 2012;Ottolini et al., 2011;Sturpe and Schaivone, 2014) but is an underused tool in assessing clinical teaching skills (Fakhouri Filho and Nunes, 2019). In a study of 20 clinical faculty members who participated in the faculty development on professionalism and medical ethics in the USA, Lu et al. (2014) found that faculty confidence and attitudes regarding teaching professionalism were significantly increased (p < 0.05) after participating in the OSTE workshop (Lu et al., 2014). Similarly, in a study conducted by Julian et al. (2012) of 46 faculty members who participated in a faculty teaching skills workshop in the USA, the faculty reported statistically significant improvements in self-assessed teaching skills after the OSTE. Thus, faculty satisfaction with the OSTE experience was high (Julian et al., 2012). In a study of 14 hospitalists who participated in a faculty development program, Ottolini et al. (2011) found that pre-and post-OSTE variables for the four stations (optimize orientation, feedback, clinical reasoning, assessing physical examination findings, and promoting resident leadership) were statistically significant (p < 0.0001) (Ottolini et al., 2011). Based on the literature review, we found that most OSTErelated research studies investigated the immediate effects after completing the OSTE.

Methods
This study used a cross-sectional design of retrospectively collected data. Participants were recruited using purposive sampling at a medical teaching hospital in central Taiwan from April to June 2016. The inclusion criteria were N3level nurse preceptors who had completed previous OSTE training during 2013-2015. In Taiwan, there is a five-level clinical ladder system, guided by the Taiwan Nurse Association (Lee, Dai and Mccreary, 2015). The rankings are: N, novice; N1, performs general patient care; N2, participates in critical patient care; N3, executes integrated nursing care and is responsible for clinical educational activities; and N4, functions as a nursing administrator and conducts nursing research (Lee, Dai and Mccreary, 2015). In this study, N3-level nurse preceptors were responsible for training nurses who were new to the department, teaching competency in the required skills in clinical practice. In total, 42 N3-level nurse preceptors were invited to participate in this study. The response rate was 76.2% (32/42 nurses) due to turnover (six nurses), transfer to another unit (two nurses), illness (one nurse), and overseas travel (one nurse).
In this hospital, during 2013-2015, 12 tips were used to develop the OSTE (Boillat et al., 2012). The participants from the faculty development program in Nursing participated in three stations during a 1-h session, which included a 12-min section on how to provide feedback. The passing score for each station in this previous OSTE was 66.8 points (DOPS station), 73.3 points (mini-CEX station), and 71.5 points (CBD station). See Table 1. 9. To protect the teacher in the hot seat To prevent N3 nurse preceptors from feeling like they were in the hot seat, we designed a 4-min debriefing session at the end of each station.

To incorporate the OSTE into faculty development in this context
We incorporated the OSTE into the nurse clinical ladder system at level N3. 11. To promote buy-in Two unique aspects of the OSTE that can be highlighted are the help it can provide N3-level nurse preceptors in using the assessment tools and the immediate feedback from the debriefing with peers.
Procedure and data collection.Nurse preceptors were invited to participate in this study by telephone. The first author held a 10-min one-to-one interview with each participant who agreed to take part in the study. During the interview, a semi-structured questionnaire was used for collecting the data. The questionnaire comprised three parts: demographic, teaching behavior, and OSTE-related variables. The demographic variables included age, length of service, marital status, educational level, whether they have children (i.e., responsibilities that could affect clinical teaching), and work unit. As most of the nurse preceptors in this study were female, sex was not included as a variable. The teaching behavior variables included the use of the DOPS, mini-CEX, and CBD, as well as the use of the sandwich feedback technique when using these assessment tools. The OSTE-related variables consisted of the mean scores (checklist and global rating) recorded by the evaluators in the previous OSTE training during 2013-2015.
Statistical analysis. All analyses were conducted using IBM ® SPSS ® English version 20.0 (IBM, 2018). We used descriptive analyses to examine the demographic, teaching behavior, and OSTE-related variables. Correlation analyses were performed to assess the relationships between teaching behaviors and OSTE-related variables.

Results/Analysis
Descriptive analyses of the demographic, teaching behavior, and OSTE-related variables. The mean age of nurse preceptors was 33.3 years (standard deviation: ± 3.8 years), with an average seniority of 11.0 years (standard deviation: ± 3.1 years). The majority of participants were educated to the level of a bachelor's degree (87.5%), were single (59.4%), and without dependents (56.3%). The majority of the nurse preceptors worked in the intensive care unit (43.7%) and surgical-medical wards (34.4%). Our clinical teaching evaluation showed that a high percentage (90.6%) of nurse preceptors used the DOPS assessment tool, more than half (59.4%) used the mini-CEX assessment tool, and 40.6% used the CBD assessment tool. The nurse preceptors who used all three assessment tools (40.6-90.6%) also used the sandwich feedback technique when they taught novice nurses during their clinical education. The mean scores for each OSTE station were: 82.9 points out of a passing score of 66.8 points for the DOPS station, 71.0 points out of a passing score of 73.3 points for the mini-CEX station, and 69.4 points out of a passing score of 71.5 points for the CBD station during the OSTE in 2013-2015. See Table 2. 2) * OSTE, objective structured teaching evaluation. ** SD, standard deviation. *** DOPS, direct observation of procedural skills.
Correlation analyses. An analysis of the correlations between the variables showed that use of all three assessment tools by a nurse preceptor was correlated with the use of the sandwich feedback technique (r = 0.94-1, p < 0.01). The scores on the previous OSTE stations had no statistical correlations with the use of the DOPS assessment (r = 0.25, p > 0.01), mini-CEX assessment (r = 0.16, p > 0.01), or CBD assessment (r = −0.06, p > 0.01) tools. Meanwhile, there was no correlation between the previous OSTE scores and use of the sandwich feedback techniques (r = 0.05-0.25, p > 0.01). See Table 3. 9. CBD station scores * OSTE, objective structured teaching evaluation. ** DOPS, direct observation of procedural skills. *** Correlation is significant at the 0.01 level (2-tailed). **** mini-CEX, mini-clinical evaluation exercise. ***** CBD, case-based discussion.

Discussion
Nurse preceptors used workplace-based assessment tools and the sandwich feedback technique. In this study, nurse preceptors reported that they use the DOPS assessment (90.6%), mini-CEX assessment (59.4%), and CBD assessment tools (40.6%). The results are consistent with the idea that one of the roles of nurse preceptors is to act as an assessor (Harden and Crosby, 2000;Joint Commission of Taiwan, 2018), and nurse preceptors continued to use workplace-based assessment tools in the clinical environment. Additionally, nearly 90% of nurse preceptors used the DOPS assessment tool. This high percentage may be attributed to a policy of this hospital stating that novice nurses had to acquire six nursing skills using the DOPS assessment tool evaluated by nurse preceptors. These six nursing skills were sputum suction, use of an infusion pump, intravenous injection, use of an indwelling catheter, blood transfusion, and oral administration of medication. During this study, approximately 30 novice nurses joined this teaching hospital each month, and the nurse preceptors used DOPS assessment tools to evaluate the skills of the novice nurses during the probation period.
We also found that nurse preceptors who used all three assessment tools in the evaluation of clinical teaching also practiced the sandwich feedback technique. Feedback skills are important in clinical teaching because they assist in the retention of knowledge by the nurses (Foster, 2015). Teaching behavior variables and previous OSTE scores. In this study, we did not find correlations between the teaching behavior variables (use of the three assessment tools and sandwich feedback technique) and the previous OSTE scores the nurse preceptors received during 2013-2015 (p > 0.01). This may indicate that clinical teaching behaviors are more dependent on hospital teaching policy rather than on OSTE scores. Therefore, nurse educators should develop learning activities for planning and implementation at the beginning of the faculty development program, such as the OSTE, to demonstrate the value of implementing new teaching skills in the workplace (DeSilets, 2018).
Limitation. This study was conducted at a single teaching hospital; hence, these findings are not representative of all nurses in Taiwan. Further investigations are warranted to analyze the effects of outcomes after the OSTE on the teaching behavior of nurse preceptors as well as on patient care outcomes.

Conclusion
This study illustrated the good teaching behaviors the nurse preceptors exhibited using workplace-based assessment tools and the sandwich feedback technique. The previous OSTE scores were not correlated with teaching behaviors. These results may facilitate the clinical education role and cultivate teaching abilities. The clinical teaching behavior may be more of hospital teaching policies than of the OSTE training scores.

Shu-Chen Wu is a BSN and RN. She is a head nurse of the Nursing Department in the Changhua Christian
Hospital. She is also a member of the nursing education committee and she is in charge of the education program in pediatric units.

Yung-Sung
Wen is a MHA and MD. He is a director of the Faculty Development Center and an ENT surgeon in the Changhua Christian Hospital.