Evaluation of an evidence-based practice training for peer support workers in behavioral health care

Abstract Service provision by peer support workers (PSWs) is growing within the US behavioral health care system and research is needed to determine appropriate training and support. This study examined responses of 15 PSWs and 20 licensed behavioral health practitioners (BHPs) who participated in a one-day training of seeking safety (SS). PSWs and BHPs provided post-training feedback on satisfaction with the training and their perceived comfort level in implementing SS. Overall, PSWs and BHPs reported high satisfaction and comfort, and except for one significant difference, PSWs benefited from the training to the same extent as BHPs. Compared to BHPs, PSWs reported significantly greater improvement in their counseling ability within the SS model as a result of the training. Considering the varying levels of education and training within the field of PSWs, our results suggest that a “one-size fits all” approach to professional training may be acceptable; however, further research is warranted.

ABOUT THE AUTHOR Dr A.S. Crisanti is an associate professor at the University of New Mexico, Department of Psychiatry and Behavioral Sciences. Her research aims to increase access and retention to behavioral health treatment by expanding peer-delivered services. The research reported in this paper was part of a large comparative effectiveness study funded by the Patient Centered Outcomes Research Institute (PCORI CE-12-11-4484) on the effectiveness of peerdelivered services for post-traumatic stress disorder and substance use. In the PCORI funded study, peer-support workers (PSWs) delivered the evidence-based practice Seeking Safety. All PSWs were provided training for the EBP along with other behavioral health practitioners with more education, experience and training in mental illness and addictions. The authors goal was to determine whether a one-size fits all approach to training was appropriate or if PSWs needed customized training. The research reported addresses the professional development of PSWs and their better integration in the workforce.

PUBLIC INTEREST STATEMENT
The number of peer support workers in behavioral health care systems has increased in the United States in the past 20 years. While the certification process for PSWs varies from state to state, training is limited and because it is restricted to people with lived experience, the curricula is tailored to their literacy, education, and experience levels. Professional development opportunities have been identified to be important to PSWs with almost 90% reporting that additional training in special topics would improve their professional experience. The purpose of this evaluation study was to determine whether PSWs, once in the workforce, need separate training from their colleagues, especially given less training and education in behavioral health compared to behavioral health practitioners, such as social workers, psychologists, and psychiatrists. A better understanding of PSWs' training needs is necessary to inform how to increase job satisfaction and further advance their successful integration into practice.

Introduction
The number of peer support workers (PSWs) in behavioral health care systems (serving individuals with mental health problems and/or addictions) has increased in the United States in the past 20 years (Davidson, Bellamy, Guy, & Miller, 2012). There are several reasons for this growth, including the reimbursement of peer support services under the Medicaid program, a growing body of literature on the effectiveness of peer-delivered services, and an emphasis on recovery-oriented behavioral health systems (Chinman et al., 2014;Daniels, Bergeson, Fricks, Ashenden, & Powell, 2012;Davidson et al., 2012;Lloyd-Evans et al., 2014;Repper & Carter, 2011;Rogers et al., 2012;Tucker et al., 2013). The role of PSWs has also expanded over the years from offering mutual support through self-help groups and peer-run programs (e.g. drop-in centers) to the provision of services (Repper & Carter, 2011). Two examples in which PSWs have demonstrated effectiveness in the delivery of services include selfmanagement interventions, such as wellness recovery action planning (WRAP) and case management. In a randomized controlled trial of peer-led WRAP compared to treatment as usual (TAU), WRAP participants reported significantly greater reduction overtime in psychiatric symptoms and improvements in hopefulness and quality of life compared to clients receiving TAU (Cook et al., 2012). A randomized control trial of regular case management compared to peer-based case management found better treatment engagement six months after entering treatment and higher use of Alcoholics Anonymous and Narcotics Anonymous at 12 months among individuals with serious mental illness who were assigned to the team with PSWs (Sells, Davidson, Jewell, Falzer, & Rowe, 2006).
The certification process for PSWs varies from state to state and certification is only required for Medicaid reimbursable services, such as comprehensive community support services and assertive community treatment (Kaufman, Brooks, Steinley-Bumgarner, & Stevens-Manser, 2014). In New Mexico, for example, the Certified Peer Support Worker (CPSW) training requires 40 h of classroom time over 5 days followed by a certification exam through the New Mexico Credentialing Board for Behavioral Health Professionals. Some of the topics covered in the five-day training include professionalism, ethics, components of recovery and resiliency, mental health and substance use disorders (SUDs), communication skills development, and stress management.
While the certification process for PSWs varies across the states, the one common feature among the various training programs is that the training sessions are restricted to people with "lived experience" with the curricula being tailored to the literacy, education, and experience level of trainees. Once in the workforce, however, PSWs may be required to complete additional training (e.g. motivational interviewing, trauma-informed care, etc.) that will likely include individuals from various professions and academic backgrounds. If the sessions are tailored towards learners with higher educational and training backgrounds and adhere to more of a "one-size fits all" approach, professional development training may be intimidating and/or less effective for PSWs.
Professional development opportunities have been identified to be important to PSWs (Gerry, Berry, & Mayward, 2011;Simpson, Quigley, Henry, & Hall, 2014). In a recent study on the professional experiences of peer specialists, 89% reported that additional training in special topics would improve their professional experience (Ahmed, Hunter, Mabe, Tucker, & Buckley, 2015). As PSWs become more involved in service delivery, their need for professional development training will increase. Training on the implementation of evidence-based practices (EBPs) is especially critical to ensure fidelity in the delivery of the model (Gorman-Smith, 2006;Webster-Stratton, Reid, & Marsenich, 2014). Reviews on the effectiveness of EBP training programs indicate the importance of alignment of material to the needs of learners (Dizon, Grimmer-Somers, & Kumar, 2012). The effectiveness of EBP training has been evaluated for various groups, including physicians, allied health professionals, and mental health professionals, but this research has not included PSWs (Dizon et al., 2012;Taylor et al., 2000). The purpose of this evaluation study was to explore the effectiveness and acceptability of a one-day training on the implementation of a trauma-specific EBP for PSWs in contrast to "traditionally trained" behavioral health practitioners (BHPs), including social workers, psychologists, and psychiatrists. The trauma-specific EBP was seeking safety (SS), which is a present-focused cognitive-behavioral therapy designed to target trauma, post-traumatic stress disorder and/or SUD (Najavits, 2002). Notably, SS is the only model thus far for this comorbidity that has already accrued some evidence in peer-led format (Najavits et al., 2014).

Methods
A post-training evaluation design was used with data collected immediately following completion of a one-day six-hour training on SS. The training focused on teaching participants of various training and educational backgrounds how to implement the EBP. Dr Najavits, author of the SS book, conducted the training, which involved the standard one-day SS training that has been conducted by her team for many years (http://www.treatment-innovations.org/seeking-safety.html). The goals of the SS training were to: (1) review current understanding of evidence-based treatment of trauma and/or substance abuse; (2) increase empathy and understanding of trauma and substance abuse; (3) describe SS; (4) provide assessment and treatment resources; and (5) identify how to apply SS for specific populations, such as homeless, adolescents, criminal justice, HIV, military/veteran, etc. In addition, the training was designed to inspire confidence in using the EBP, to provide an opportunity for trainees to discuss implementation issues and to conduct various experiential learning exercises, including conducting a role-play of a full SS session. Thirty-seven individuals completed the SS training: 16 PSWs and 21 BHPs, with the latter comprising various behavioral health professionals, including social workers, psychologists, and psychiatrists and addiction counselors.
A survey developed specifically for the training was used to collect feedback from trainees. The brief survey included questions on demographics (i.e. gender and ethnicity/race) training content, training delivery, and experience with SS prior to the training. Changes in knowledge and skill attainment (i.e. training content) were assessed through the five questions shown in Table 1. For each question, trainees were asked to provide two ratings: a retrospective estimate and a post-training assessment. Changes between the two estimates were examined in the analysis.
Three questions on training delivery determined satisfaction with training, comfort level with asking questions, and level of difficulty of the material covered during the training. Using a four-point Likert-type scale, ranging from strongly agree to strongly disagree, trainees were asked to indicate: (1) how comfortable they felt asking questions, (2) the extent to which they understood all of the material that was presented, and (3) how satisfied they were with the material they learned in the training. A final question related to training delivery asked about whether the amount of material covered during the training was too much, too little, or just the right amount.
Four questions focused on experiences with SS prior to the one-day training. Trainees were asked to indicate whether they: (1) ever attended a SS training, and if so, for how many hours; (2)  implementing the practice. The SS Training DVDs (totaling 4.5 h) provide an overview of the model, an example of a session conducted with real clients, a demonstration of grounding, and a fidelity session. The survey concluded with an open-ended question that encouraged trainees to list any additional feedback, including aspects of the training they found to be most useful, challenges that they might face in implementing SS, and whether there was a need for additional SS-related training.

Analysis
Descriptive statistics were calculated to summarize the demographic, background, and responses to the survey items on training content and delivery. Medians and quartiles were calculated for Likert scales and change scores, and frequencies and percentages for categorical variables. We also calculated chi-square or Fisher exact tests to compare participant demographics between PSWs and BHPs and Wilcoxon rank-sum tests to compare change scores on the items related to training content (e.g. skill attainment and knowledge acquisition) and training delivery (e.g. satisfaction and comfort level between the two groups of trainees).
We adjusted for multiple comparisons between the variables related to training content and delivery via a Bonferroni correction such that two-sided p-values were considered significant if ≤ 0.01 (α = 0.05/5). We performed ordinal logistic regression on two of the change scores of particular interest related to training content (i.e. ability to counsel and manage clients about the topics covered in the training) to assess whether PSWs felt as comfortable implementing the SS program as BHPs after controlling for previous exposure to SS material (reading SS books, watching SS videos, previously conducted SS sessions with clients, or previously attended SS training).

Institutional review board
This study met the requirement for exemption in that it involved normal educational practices, specifically research on the effectiveness of the SS curriculum.

Results
Thirty-five participants completed the questionnaire resulting in a 95% response rate comprising 15 PSWs and 20 BHPs. The BHPs included 13 addiction counselors (65%), 4 social workers (20%), 2 psychologists (10%), and 1 psychiatrist (5%). Both genders were equally represented: 51% males and 49% females. The ethnic/racial composition of the group was diverse among the groups. PSWs were more likely than BHPs to be Hispanic (56 vs. 26%) and BHPs were more likely than PSWs to be White (79 vs. 20%). Except for the extent to which DVDs were watched, there were no significant differences between PSWs and BHPs in terms of experience with SS prior to the training, including reading the SS manual, structured training, or implementation of the EBP. BHPs were significantly more likely than PSWs to have watched at least some of the SS DVDs prior to the training: 33 vs. 89%, respectively (χ 2 = 12.22, p < 0.01).
Significant change scores for PSWs and BHPs were observed for all five items that asked about perceived effectiveness of training with respect to ability to implement SS and knowledge of the subject matter (i.e. training content). Table 2 summarizes the median, first, and third quartile ratings for two time points (retrospective assessment of before and after), and for the change between time points for PSWs and BHPs.
When change scores for PSWs were compared to change scores for BHPs, only one significant difference was observed (see third section Table 2). PSWs reported a significantly greater improvement in their ability to counsel clients with SS as a result of the training compared to BHPs (Wilcoxon rank sum test (S = 351.00, p < 0.01).
Overall, PSWs and BHPs equally reported being satisfied with the delivery of the training, feeling comfortable asking questions about the training and understanding all of the material that was presented during the training. There were also no differences between PSWs and BHPs in terms of how they felt about the amount of material covered during the training. Sixty percent of the PSWs and 65% of the BHPs reported that the amount of material covered was just right.

Table 2. Perceived effectiveness of training for PSWs and BHPs
*p-value is significant if ≤ 0.01. Type I error rate was Bonferroni-corrected to adjust for multiple comparisons. 1 Scaling is low 1-2, medium 3-4, and high 5. We were particularly interested in assessing PSWs and BHPs' comfort with counseling clients and implementing the SS program. Therefore, two ordinal logistic regression models were fitted to the change scores for each of these outcomes. We controlled for background exposure to the SS including previous reading of the SS book, watching the SS DVDs, number of previously conducted SS sessions, attendance at previous SS training, and demographic characteristics. We found that after controlling for these covariates, the improvement in comfort with counseling and implementing the SS program did not differ significantly between PSWs and BHPs (Wald test: χ 2 = 0.12, p = 0.73; χ 2 = 0.82, p = 0.37, p = 0.36, respectively).

Discussion
SS is an EBP for trauma and/or SUD that has been used for over 20 years in a wide variety of settings and with diverse clients. Recently, SS has also evidenced positive outcomes when delivered by PSWs (Najavits et al., 2014). This study sought to evaluate how PSWs vs. BHPs responded to the standard SS training. Would they report comparable satisfaction with the training? Comparable learning? Comparable comfort with delivering SS? With the growing recognition of PSWs as an asset in the delivery of behavioral health care, better understanding of their training needs can help inform how to further advance their successful integration into practice. This is the first study to evaluate the impact of training of PSWs vs. BHPs on SS.
The evaluation followed a one-day standard training on SS. In addition to a direct comparison of PSWs vs. BHPs on all items, the impact of prior exposure to SS that may have influenced responses was also accounted for. Both PSWs and BHPs reported significant improvements on survey items from pre-to post-training, indicating that they perceived the training as beneficial. Furthermore, there were no differences between PSWs and BHPs on most survey items and in the impact of prior exposure to SS. These findings are positive news. They can be interpreted as indicating the PSWs and BHPs can attend the same SS training, which can make for an efficient and less costly method than having to provide separate training for each. Moreover, being trained together can help build positive connections between PSWs and BHPs that can infuse their work-encouraging mutual respect and common understanding. Only one item evidenced a difference between PSWs and BHPs. PSWs reported a significantly greater improvement in their ability to counsel clients with SS as a result of the training compared to BHPs. This is a highly positive finding in that PSWs by definition started out with less training and education in behavioral health than BHPs and reported a greater increase in their ability to use a behavioral health model such as SS. For many peers, this was likely their first exposure to training in how to deliver an EBP and it was encouraging to see that their comfort level with doing so increased so rapidly. Perhaps reflecting their excitement about the opportunity for professional development training, in contrast to the BHPs, every PSW responded to the open-ended question on the survey. PSWs indicated an appreciation for the background information presented on the research behind the model. Role playing, grounding exercises, videos, and practice sessions were also noted by PSWs as being some of the most useful aspects of the training.

Limitations
This was an initial, exploratory study, and thus results need to be further replicated. The sample was relatively small, and thus the absence of differences could simply reflect a lack of statistical power. The small sample also limits generalizability of the findings. Some information, such as years' experience among trainees in providing clinical services, was not available; however, clinicians' experience level is not known to impact outcomes so it is unclear how important such data might be (Najavits & Weiss, 1994). The retrospective estimate of pre-test knowledge at the end of the one-day training was also a limitation, and our results would have been strengthened had trainees completed the questions on training content prior to the onset of the training to reduce the influence of recall bias. While pilot-tested before implementation, the post-training survey was developed specifically for this training with no established psychometric properties. Our study was also limited in that we did not have the capacity to conduct a follow-up survey months after the training to evaluate whether the results at post-training held when PSWs and BHPs were actually out in the field implementing SS. It could be that initial confidence and comfort with SS by the end of training may have changed in the context of implementation challenges with real clients. We also did not conduct a knowledge test of SS principles and thus relied on their perceptions of learning the model. While research in health promotion indicates that self-efficacy beliefs are related to health behavior outcomes, we do not know whether knowledge and skills acquired from the SS training actually translates into action (Bandura, 2004). Thus, future research would benefit from a follow-up period after implementation of the model as well as an objective knowledge test of SS principles.

Conclusions
Overall, our results add further support to the increasing focus on PSWs in the delivery of behavioral health care and in particular to the feasibility of SS as a model that peer facilitators have responded to positively both in this study and in a prior pilot-outcome study on peer-led SS (Najavits et al., 2014). Given the sheer numbers of traumatized individuals, as well as issues of SUD that are prominent in this population, a model such as SS delivered by PSWs and BHPs, can add to the goal of improving care for this vulnerable segment of the population. It is heartening to see that PSWs can come to feel comfortable and positive about a trauma-informed model on a par with professional counterparts, even within as short a time as a one-day training. However, it is important to note that some of the trainees, including both PSWs and BHPs, wrote on the survey that they would benefit from becoming more familiar with SS through their own study of the DVDs, SS materials, and general practice before being fully comfortable with implementing the EBP. Implementation research has shown the value of ongoing training in the successful delivery of EBPs, especially as it relates to fidelity and this may be particularly true for PSWs who may be implementing an EBP for the first time (Webster-Stratton et al., 2014).

Funding
This work was supported through a Patient-Centered Outcomes Research Institute (PCORI) Program Award ] to the first author; and a PCORI Eugene Washington Engagement Award [0054-BVARI] to the fifth author.