Effectiveness of Evidence-Based Practice (EBP) Education on Emergency Nurses’ EBP Attitudes, Knowledge, Self-Efficacy, Skills, and Behavior: A Randomized Controlled Trial

Background: Emergency care clinicians are expected to use the latest research evidence in practice. However, emergency nurses do not always consistently implement evidence-based practice (EBP). An educational intervention on EBP was implemented to promote emergency nurses’ use of EBP, and the effectiveness of it was evaluated. Aims: This study aimed to evaluate the effectiveness of an EBP educational intervention on emergency nurses’ EBP attitudes, knowledge, self-efficacy, skills, and behavior. The study also examined learners’ satisfaction with the EBP educational intervention. Methods: A randomized controlled trial with parallel groups with evaluations before the education, immediately after it, and 6 and 12 months after the education was conducted at four emergency departments in two university hospitals. The experimental group ( N = 40) received EBP education while the control group ( N = 40) completed self-directed EBP education. The primary outcomes were emergency nurses’ EBP attitudes, knowledge, self-efficacy, skills, and behavior, while the secondary outcome was satisfaction with the EBP education. Results: Thirty-five participants of an experimental and 29 participants of a control group completed the study. There were no statistically significant ( p < .05) improvements and differences between groups in EBP attitude, self-efficacy, or behavior immediately after the EBP education. At the 6-month measurement point, the experimental group showed significantly better EBP attitudes, behavior, knowledge, and self-efficacy than the control group. At the 12-month measurement point, the improvements began to decrease. The groups also differed significantly in terms of participant satisfaction with how the teacher encouraged learners to ask clinical questions. a positive effect on emergency nurses’ EBP attitudes, knowledge, self-efficacy, skills, and behavior. The effects of the education appeared the best 6 months after the education. After this point, the results began to decrease and approached baseline levels. EBP educational interventions designed for emergency nurses should apply various teaching strategies to improve their EBP attitude, knowledge, self-efficacy, skills, behavior, and satisfaction with the education.


Effectiveness of EBP Educational Intervention
. Educational interventions on EBP is one strategy to promote EBP use among nurses (Häggman-Laitila, Mattila, & Melender, 2016;Saunders, Vehviläinen-Julkunen, & Stevens, 2016). To assess the impact of EBP education, the Classification Rubric for EBP Assessment Tools in Education (CREATE) framework was developed. Seven categories for assessing the outcomes of EBP education are provided by CREATE (Tilson et al., 2011), namely, learners' reactions to the EBP educational experience, attitudes, self-efficacy, knowledge, skills, behavior, and benefits to the patient. Reaction to the EBP educational experience describes learners' opinions regarding the learning experience and the intervention's efficacy, from here on referred to as learners' satisfaction. Attitudes encompass learners' perceptions of the importance and usefulness of EBP in informing clinical decision-making. Self-efficacy refers to learners' judgments of their ability to perform a given activity, while knowledge describes learners' awareness and understanding of EBP concepts. Skills relate to the application of knowledge and describe an individual's ability to perform the EBP steps in a practical setting. Behavior describes learners' real-life actions, that is, their commitment to following the EBP steps in everyday practice. It includes all the processes used in the application of EBP. Benefits to patients cover how the EBP educational intervention will impact the health of patients and communities. This category is difficult to measure, as the EBP learning process is affected by numerous variables. Effective EBP education clearly addresses all these learning outcomes (Tilson et al., 2011).
Several studies have investigated how an educational intervention affects nurses' general attitudes toward EBP. An online EBP course with a self-paced PowerPoint presentation (Moore, 2017), a multifaced EBP education for registered nurses (RN) that included mentoring (Friesen, Brady, Milligan, & Christensen, 2017), and an EBP course for Master of Science in Nursing and Doctorate of Nursing Practice students (Moore, Watters, & Wallston, 2019) had positively impacted nurses' EBP attitudes, which subsequently significantly affected EBP implementation (Friesen et al., 2017;Moore et al., 2019). In contrast, Ramos-Morcillo, Fernández-Salazar, Ruzafa-Martínez, and Del-Pino-Casado (2015) reported that a brief, 30 hours of online and 10 hours of face-to-face learning, educational intervention on EBP had no impact on professional nurses' attitudes.
EBP knowledge and skills are often evaluated together. A brief, 30 hours of online and 10 hours of face-to-face learning, EBP course (Ramos-Morcillo et al., 2015) for professional nurses and EBP education that included mentoring for RNs (Saunders et al., 2016) significantly improved their knowledge and skills. Moore et al. (2019) found that an EBP course increased only Doctorate of Nursing Practice students' knowledge. On the other hand, an online EBP course did not significantly improve RNs' knowledge or skills (Moore, 2017).
Hence, previous research has already extensively studied the effectiveness of EBP educational interventions. However, to the best of our knowledge, no research has focused on the effectiveness of educational interventions for emergency nurses that explicitly include the EBP steps as learning content.

AIM
This study aimed to evaluate effectiveness of an EBP educational intervention on emergency nurses' EBP attitudes, knowledge, self-efficacy, skills, and behavior. The study also examined learners' satisfaction with the EBP educational intervention.

Study Design
This study employed a randomized controlled trial (RCT) with a parallel group and a 12-month follow-up period. The RCT was conducted in four emergency departments at two university hospitals in Finland, and in the randomization of the units, they were allocated to an experimental group or a control group by an impartial secretary using a simple coin toss.

Setting and Participants
The first author recruited the study participants. The study population comprised 300 emergency nurses employed in the two university hospitals' emergency departments. Three sets of inclusion criteria were used: (a) working as an RN at a university hospital emergency department; (b) working full-or part-time as an RN at a university hospital emergency department; and (c) ability to understand and read Finnish. The sample size necessary for identifying statistically significant differences was estimated using a simulation approach. Various expected EBP improvements (5% to 20% for a half and a third of the participants) were bootstrapped 1,000 times to produce sample sizes. The power was approximated by Original Article the number of significant Mann-Whitney U tests when comparing increases in groups and Wilcoxon signed-rank tests when comparing increases on the individual level. As the study included repeated measurements, drop-out attrition was expected. Assuming a 20% attrition rate, a sample size of 80 participants was required to achieve 80% power and an alpha level of .05. The 20% attrition rate was assumed based on earlier research (Julious, 2010). At the beginning of the study, 80 participants were recruited (40 for the experimental group and 40 for the control group). A total of 35 and 29 participants from the experimental and control groups, respectively, completed the study (Figure 1).

The Educational Interventions
The experimental group received an intervention named "Evidence-Based Practice Basics for Emergency Nurses" (EBP Basics EmNurs; two European Credit Transfer System [ECTS]), while the control group completed "Self-Directed Learning Module: Evidence-Based Practice Basics for Emergency Nurses" (Self-Dir EBP Basics EmNurs; two ECTS). The main objective for both interventions was to provide emergency nurses with basic EBP competencies. The key differences between the groups were that the EBP education for the experimental group included multifaceted educational strategies, such as didactic lectures and discussions, small group tutorials, database search workshops, Effectiveness of EBP Educational Intervention and self-directed learning (i.e., assignments and reading). The education for the control group included self-directed learning (i.e., reading and database searching) as an educational strategy. Descriptions of the educational interventions according to the GREET checklist (i.e., the Guideline for Reporting Evidence-Based practice Educational interventions and Teaching; Phillips et al., 2016) are presented in Table S1.

Outcomes and Instruments
The primary and secondary outcomes were measured according to the CREATE framework (Tilson et al., 2011). Data were collected using five questionnaires. Three of the five questionnaires had been previously tested and validated: (a) The Evidence-Based Practice Questionnaire-FI (EBPQ-FI; Upton & Upton, 2006); (b) Evidence-Based Practice Beliefs Scale© (EBPB; Melnyk, Fineout-Overholt, & Mays, 2008); and (c) Evidence-Based Practice Implementation Scale© (EBPI; Melnyk et al., 2008). The knowledge test and the course assessment form (i.e., learners' satisfaction) were developed by the researcher, as no validated tests or forms appropriate for the purposes of this study were found. The knowledge test questions were based on the EBP competencies presented by Melnyk, Gallagher-Ford, Long, and Fineout-Overholt (2014) so that there was one question regarding each competency. The course assignment form questions included in the instrument were based on items proposed by Tilson et al. (2011). Both instruments were trialed when piloting the educational intervention. The results were analyzed statistically to examine the measurement properties of the instruments. More details about the instruments are presented in an earlier article (Koota, Kääriäinen, Lääperi, & Melender, 2019).

Data Collection
Data were collected from participating emergency nurses through an electronic and manual survey, administered at four time points (T₀-T₃) during May 2018 through August 2019. The pre-intervention survey (T₀) was administered in May 2018, while T₁ was administered immediately after the educational intervention (at the completion of the education). The T₂ was administered after 6 months of T₁, and T₃ was administered after 12 months of T₁.

Data Analysis
EBP attitudes, self-efficacy, knowledge, skills, and behavior sum variables (developed by the researchers) were formed by calculating the means within a category. Scores for the questions "I believe that EBP takes too much time" and "I believe EBP is difficult" were reversed to match the scale (negative-to-positive) of other questions included in the attitude EBP sum variable. Continuous variables are presented as means and medians, while categorical data are presented as frequencies and percentages. The categorical variables were analyzed with a Chi-squared test. Continuous variables in two different categories were compared using Mann-Whitney U test and Kruskal-Wallis H test when more than two categories were present. Pearson's correlation coefficients were also calculated. Two-tailed p values <.05 were considered statistically significant. The analyses were performed using R (R Core Team, 2014), and figures were produced using the package ggplot2 (Wickham, 2016).

Ethical Considerations
The study was approved by the University Hospital Ethical Board (an institutional review board). Permission to conduct the study was also obtained from both university hospitals. Written consent was obtained from every participant after they had received oral and written information about the study. The information encompassed the voluntary and anonymous nature of participation. Participant names were replaced with a code number in all documents.

RESULTS
Altogether, the emergency nurses (N = 80) were randomized into two groups (i.e., the experimental group and the control group). Thirty-five participants in the experimental group and 29 participants in the control group completed the educational intervention ( Figure 1). There were no statistically significant between-group differences identified at T 0 in EBP attitudes, self-efficacy, skills, or behavior. However, the knowledge test in the experimental group obtained statistically significant lower values than the control group. Findings concerning the primary outcomes are outlined in Table 1. At each measurement point, between-group differences in the mean value of each sum variable (p group ) along with the difference between the baseline and specific measurement point mean value for both groups (p delta ) were assessed.
Immediately after the intervention (T 1 ), there was a statistically significant difference between the groups in EBP behavior according to EBPI results, but this difference was not identified from EBPQ results. However, the experimental group showed statistically significant improvements in the knowledge test at all measurement points.
At T₂, the experimental group significantly differed from the control group in every sum variable. The experimental group showed statistically significant improvements in EBP attitudes (EBPB, EBPQ), behavior (EBPI), knowledge (EBPQ, Knowledge test), skills (EBPQ), and self-efficacy (EBPB) relative to baseline measurements ( Table 1). The control group had a statistically significant increase in EBP attitude (EBPB) and self-efficacy. At the same time, EBP behavior (EBPQ) decreased.
A statistically significant difference between the groups in EBP attitudes (EBPB) was identified at T₃, but this difference was not observed from EBPQ results. The experimental Original Article group had improved more in EBP knowledge (measured with both instruments) and in self-efficacy 12 months after the intervention than the control group (Table 1).
Learners' satisfaction with the educational intervention is summarized in Table 2. The groups differed significantly in satisfaction with the teacher's ability to encourage nurses to ask clinical questions. Furthermore, the groups significantly differed in satisfaction concerning the usefulness of consideration on how to use research evidence in nursing practice.  The correlations between participants' background data and the sum variables at T 0 are reported in another article (Koota et al., 2019). These correlations stayed constant throughout the study (Table S2).

DISCUSSION
To the best of our knowledge, this is the first study that evaluates emergency nurses' EBP attitudes, behavior, knowledge, self-efficacy, and skills before and 12 months after an EBP educational intervention. In addition, learners' satisfaction with the education was analyzed.

Evidence-Based Practice Attitudes
Both groups showed better than average baseline EBP attitudes. In the control group, EBP attitudes stayed at a constant level throughout the follow-up period according to the results from both EBP attitude instruments. In the experimental group, EBP attitudes improved at T 2 , but then decreased at T 3 , although EBP attitudes were still higher than T 0 (EBPB measurements). Earlier studies (Friesen et al., 2017;Moore et al., 2019;Moore, 2017;Ramos-Morcillo et al., 2015;Snibsøer et al., 2017) have noted a more prominent improvement in EBP attitudes immediately after education, but they did not include a long follow-up period.

Evidence-Based Practice Behavior
Interestingly, EBP behavior at T 0 was lower than the average level measured with the EBPI instrument but higher than the average measured with the EBPQ instrument in both groups. In the control group, EBP behavior initially stayed constant but decreased below baseline levels by T 3 (measured with both instruments). In the experimental group, EBP behavior had improved at T 2 (measured with both instruments). However, by T 3 , it had decreased (measured with EBPB). Previous research shows conflicting findings with some reporting that EBP behavior decreased following education (Friesen et al., 2017;Moore, 2017;Snibsøer et al., 2017) and another reporting that EBP behavior increased immediately after education (Moore et al., 2019). One reason for the decrease of self-assessed EBP behavior might be that the learners may have been self-critical after better understanding the true essence of EBP during and after the education. Squires et al. (2019) studied different clinical settings and found contextual attributes to affect healthcare professionals' implementation of EBP. Furthermore, resources were found to be highly important to EBP implementation in hospital emergency rooms, which implies that emergency room staff should be provided with the relevant resources if EBP implementation is to be improved.
The better EBP behavior in the experimental group might be explained by the fact that the group received contact learning. During it, there were many discussions Assess how deeply the appraisal of research evidence was studied in relation to your learning needs?  Original Article and reflections on EBP. This might have supported the real EBP behavior.

Evidence-Based Practice Knowledge
The experimental group demonstrated higher levels of selfassessed EBP knowledge than the control group throughout the follow-up period (T 0 -T 3 ). In the control group, EBP knowledge stayed constant throughout the follow-up period. In the experimental group, the knowledge level was better at T 2 . The objective EBP knowledge test revealed that the control group had higher EBP knowledge levels at T 0 than the experimental group, with EBP knowledge staying constant throughout the study. In the experimental group, the knowledge test scores improved after the education and stayed at a high level 12 months after the education. This result agrees with what was reported by Ramos-Morcillo et al. (2015), although their measurements represent EBP knowledge at T 1 . Moore (2017) and Saunders et al. (2016) had different results, as they found EBP education to have no observable effect on nurses' EBP knowledge. Better EBP knowledge in the experimental group might be explained again by the fact that the group received contact learning. During it, there were many discussions and reflections on EBP. This might have supported deep learning (Mezirow, 2003).

Evidence-Based Practice Self-Efficacy
EBP self-efficacy improved with the control group at T 1 . In the experimental group, EBP self-efficacy increased at T 2 but then decreased to baseline levels by T 3 . A similar trend for self-efficacy (an initial increase followed by a decrease) has also been previously observed (Friesen et al., 2017;Moore et al., 2019;Royer et al., 2018;Snibsøer et al., 2017). Activating learners to self-assess their progress is an ipsative evaluation method that supports learner selfefficacy (Atjonen, 2007), and it was also used in this educational intervention. However, more methods are needed for future research to intentionally support self-efficacy of the learners during the education. For example, giving formative evaluation during the learning process (not only summative at the end) and using other methods of ipsative evaluation. It might be useful for the learners to also receive formal evaluations on their clinical EBP practices after the education (Atjonen, 2007).

Evidence-Based Practice Skills
In the control group, self-directed EBP education had no effect on emergency nurses' EBP skills. In the experimental group, nurses demonstrated minor improvement by T 2 . Previous educational intervention studies show large variation concerning EBP skills; namely, Ramos-Morcillo et al. (2015) reported that education improves EBP skills, while Moore (2017) reported decreased skill levels following an intervention. Hence, there is a clear need for developing the methods used to teach EBP skills. For example, most educational approaches have not focused on using a clinical perspective to develop EBP skills (Horntvedt, Nordstein, Fermann, & Severinsson, 2018). This aspect should be carefully considered when developing EBP education.

Learners' Satisfaction With Education
Few studies have discussed how satisfied emergency nurses are with EBP education (Koota, Kääriäinen, & Melender, 2018). In our study, nurses in both groups evaluated the learner's satisfaction with the education as above average. Jordan and Moore-Nadler (2014) and Habich and Letizia (2015) reported emergency nurses to be satisfied with the EBP education they received. However, the educational interventions of these two studies differed from the approach used in our study; thus, the results are not comparable. In our study, the experimental group, when compared with the control group, provided more positive ratings concerning the teachers' teaching style-for example, encouraging participants to ask clinical questions and to assess the usefulness of consideration on how to use research evidence in nursing practice. This finding is probably explained by the fact that control group participants were only encouraged to ask clinical questions and to consider how to apply research evidence based on written material.

STUDY LIMITATIONS
There is a lack of higher evidentiary design EBP studies reporting educational interventions in the field of emergency nursing (Koota et al., 2018). The strength of this study was that it applied an RCT with experimental and control groups. However, controlling the confounding outcomes among emergency nurses during a 12-month follow-up period is impossible. Despite an inability to control confounders, the long follow-up period was a strength, as the results provided a unique perspective on emergency nurses' EBP learning outcomes after an educational intervention and how various aspects of EBP changed over 1 year. To insure the reliability of the findings, we performed a power analysis to calculate an adequate sample size; however, there were more dropouts during the 12-month study period than we had anticipated. Due to the large number of dropouts, a sensitivity analysis was performed using only nurses who had participated at the last measurement point. These results did not differ from the results of the full data analysis, which suggested that the dropouts did not bias the results. The CONSORT list was used to insure rigorous reporting of the research.
A further strength was that three validated and reliable instruments (EBPQ, EBPB, and EBPI) were used. Moreover, since they are self-assessment instruments, it is possible that the participants may have exaggerated their EBP attitudes, skills, self-efficacy, behavior, and knowledge, a phenomenon that was reported in an Effectiveness of EBP Educational Intervention earlier study (Yost, Ciliska, & Dobbins, 2014). However, the research also applied an objective knowledge assessment instrument.
The results might be generalizable to similar populations. Our study focused on emergency department nurses; hence, the results may not be applicable to nurses working at in-patient wards. The diversity of cultural aspects should always be considered.

IMPLICATIONS FOR FUTURE RESEARCH
Based on our study, future research should focus on identifying the most effective teaching and learning method(s) for each EBP area. This is most relevant for EBP skills, as previous studies have applied various teaching and learning methods to address this EBP area but only demonstrated minor improvements (e.g., Moore et al., 2019;Snibsøer et al., 2017). Also, more research in the field of emergency nursing should examine the factors that increase EBP behavior and self-efficacy.

CONCLUSION
The tested EBP educational intervention appeared to provide the best results in the experimental group 6 months after the education. However, after 6 months, the results for most EBP areas began to decrease. By the 12-month measurement point, the results were at the baseline level or, in some cases, even lower. An exception was the knowledge test, as the participants demonstrated high levels across all measurement points.
After the educational intervention, the experimental group outperformed the control group in every measured aspect of EBP. The experimental group also demonstrated above average satisfaction with the education.
Since the results of the intervention were not permanent, future educational interventions should focus on maintaining EBP competence. More specifically, these interventions should concentrate on EBP skills and self-efficacy.
Furthermore, our results demonstrated that self-directed learning, when applied alone, was not as effective as the combination of various teaching methods-for example, didactic lectures and small group discussion. Discussions and reflections during contact learning may have an important role in supporting learners' depth learning.

LINKING EVIDENCE TO ACTION
• EBP educational interventions designed for emergency nurses should apply various teaching strategies to improve their EBP attitude, knowledge, self-efficacy, skills, behavior, and satisfaction with the education.
• The effects of this educational intervention were more positive in the experimental group 6 months after the education; after this point the results began to decrease and approached baseline levels.
• Self-directed EBP learning (control group) was not as effective as education that included various teaching strategies (experimental group).
• EBP behavior was not constant. Future educational interventions should address this challenge.