Abstract
Therapist self-efficacy in delivering evidence-based practices (EBPs) is associated with implementation outcomes, including adoption and sustainment in community mental health settings. Inner context organizational climate, including psychological safety, can proximally shape therapist learning experiences within EBP implementation. Psychologically safe environments are conducive to learning behaviors including taking risks, admitting mistakes, and seeking feedback. Organization leaders are instrumental in facilitating psychological safety, but may have differing perspectives of organizational climate than front-line therapists. Discrepant leader and therapist views of psychological safety may have independent associations with therapist EBP learning and implementation outcomes over and above average therapist perceptions of climate. This study examined survey data from 337 therapists and 123 leaders from 49 programs contracted to deliver multiple EBPs within a study examining determinants of sustainment within a large system-driven implementation. Both leaders and therapists completed measures of psychological safety climate and therapists reported on their self-efficacy in delivering multiple EBPs in children’s mental health services. Polynomial regression and response surface analysis models were conducted to examine the associations of therapist and leader reports of psychological safety and therapist EBP self-efficacy. Greater discrepancies between leader and therapist reports of psychological safety, in either direction, were associated with lower therapist EBP self-efficacy. Alignment in leader and therapist views of psychological safety climate may impact EBP implementation outcomes. Strategies for improving alignment in perceptions and priorities among organizational members can be included in organizational implementation interventions and may represent unexamined implementation mechanisms of action.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Hurlburt, M. S. (2015). Leadership and organizational change for implementation (LOCI): A randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implementation Science, 10(1), 11. https://doi.org/10.1186/s13012-014-0192-y
Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Sklar, M. (2014). Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annual Review of Public Health, 35(1), 255–274. https://doi.org/10.1146/annurev-publhealth-032013-182447
Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., Sklar, M., & Horowitz, J. (2017a). Discrepancies in leader and follower ratings of transformational leadership: Relationship with organizational culture in mental health. Administration and Policy in Mental Health and Mental Health Services Research, 44(4), 480–491. https://doi.org/10.1007/s10488-015-0672-7
Aarons, G. A., Ehrhart, M. G., Moullin, J. C., Torres, E. M., & Green, A. E. (2017b). Testing the leadership and organizational change for implementation (LOCI) intervention in substance abuse treatment: A cluster randomized trial study protocol. Implementation Science, 12(1), 29. https://doi.org/10.1186/s13012-017-0562-3
Aarons, G. A., Ehrhart, M. G., Torres, E. M., Finn, N. K., & Beidas, R. S. (2017c). The humble leader: Association of discrepancies in leader and follower ratings of implementation leadership with organizational climate in mental health. Psychiatric Services, 68(2), 115–122. https://doi.org/10.1176/appi.ps.201600062
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4–23. https://doi.org/10.1007/s10488-010-0327-7
Aranzamendez, G., James, D., & Toms, R. (2015). Finding antecedents of psychological safety: A step toward quality improvement. Nursing Forum, 50(3), 171–178. https://doi.org/10.1111/nuf.12084
Barnett, M., Brookman-Frazee, L., Regan, J., Saifan, D., Stadnick, N., & Lau, A. (2017). How intervention and implementation characteristics relate to community therapists’ attitudes toward evidence-based practices: A mixed methods study. Administration and Policy in Mental Health and Mental Health Services Research, 44(6), 824–837. https://doi.org/10.1007/s10488-017-0795-0
Beidas, R. S., Marcus, S., Aarons, G. A., Hoagwood, K. E., Schoenwald, S., Evans, A. C., Hurford, M. O., Hadley, T., Barg, F. K., Walsh, L. M., Adams, D. R., & Mandell, D. S. (2015). Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Pediatrics, 169(4), 374–382. https://doi.org/10.1001/jamapediatrics.2014.3736
Beidas, R. S., Williams, N. J., Becker-Haimes, E. M., Aarons, G. A., Barg, F. K., Evans, A. C., Jackson, K., Jones, D., Hadley, T., Hoagwood, K., Marcus, S. C., Neimark, G., Rubin, R. M., Schoenwald, S. K., Adams, D. R., Walsh, L. M., Zentgraf, K., & Mandell, D. S. (2019). A repeated cross-sectional study of clinicians’ use of psychotherapy techniques during 5 years of a system-wide effort to implement evidence-based practices in Philadelphia. Implementation Science, 14(1), 67. https://doi.org/10.1186/s13012-019-0912-4
Birken, S. A., Lee, S.-Y.D., Weiner, B. J., Chin, M. H., Chiu, M., & Schaefer, C. T. (2015). From strategy to action: How top managers’ support increases middle managers’ commitment to innovation implementation in healthcare organizations. Health Care Management Review, 40(2), 159–168. https://doi.org/10.1097/HMR.0000000000000018
Bruns, E. J., Kerns, S. E. U., Pullmann, M. D., Hensley, S. W., Lutterman, T., & Hoagwood, K. E. (2015). Research, data, and evidence-based treatment use in state behavioral health systems, 2001–2012. Psychiatric Services, 67(5), 496–503. https://doi.org/10.1176/appi.ps.201500014
Byeon, Y. V., Lau, A. S., Lind, T., Hamilton, A. B., & Brookman-Frazee, L. (2022). Organizational factors associated with community therapists’ self-efficacy in EBP delivery: The interplay between sustainment leadership, sustainment climate, and psychological safety. Implementation Research and Practice. https://doi.org/10.1177/26334895221110263
Campion, J. (2018). Public mental health: Key challenges and opportunities. Bjpsych International, 15(3), 51–54. https://doi.org/10.1192/bji.2017.11
Cronbach, L. J., & Gleser, G. C. (1953). Assessing similarity between profiles. Psychological Bulletin, 50, 456–473. https://doi.org/10.1037/h0057173
Dorsey, S., Pullmann, M. D., Deblinger, E., Berliner, L., Kerns, S. E., Thompson, K., Unützer, J., Weisz, J. R., & Garland, A. F. (2013). Improving practice in community-based settings: A randomized trial of supervision—study protocol. Implementation Science, 8(1), 89. https://doi.org/10.1186/1748-5908-8-89
Eccles, M. P., & Mittman, B. S. (2006). Welcome to implementation science. Implementation Science, 1(1), 1. https://doi.org/10.1186/1748-5908-1-1
Edmondson, A. C. (1999). Psychological safety and learning behavior in work teams. Administrative Science Quarterly, 44(2), 350–383. https://doi.org/10.2307/2666999
Edmondson, A. C. (2003). Speaking up in the operating room: How team leaders promote learning in interdisciplinary action teams. Journal of Management Studies, 40(6), 1419–1452. https://doi.org/10.1111/1467-6486.00386
Edmondson, A. C., Higgins, M., Singer, S., & Weiner, J. (2016). Understanding psychological safety in health care and education organizations: A comparative perspective. Research in Human Development, 13(1), 65–83. https://doi.org/10.1080/15427609.2016.1141280
Edmondson, A. C., & Lei, Z. (2014). Psychological safety: The history, renaissance, and future of an interpersonal construct. Annual Review of Organizational Psychology and Organizational Behavior, 1(1), 23–43. https://doi.org/10.1146/annurev-orgpsych-031413-091305
Ehrhart, M. G., & Aarons, G. A. (2022). Alignment. In F. Rapport, R. Clay-Williams, & J. Braithwaite (Eds.), Implementation science: The key concepts. Taylor & Francis.
Ehrhart, M. G., Aarons, G. A., & Farahnak, L. R. (2014). Assessing the organizational context for EBP implementation: The development and validity testing of the Implementation Climate Scale (ICS). Implementation Science, 9(1), 157. https://doi.org/10.1186/s13012-014-0157-1
Fleenor, J. W., McCauley, C. D., & Brutus, S. (1996). Self-other rating agreement and leader effectiveness. The Leadership Quarterly, 7(4), 487–506. https://doi.org/10.1016/S1048-9843(96)90003-X
Gibson, C. B., Cooper, C. D., & Conger, J. A. (2009). Do you see what we see? The complex effects of perceptual distance between leaders and teams. Journal of Applied Psychology, 94, 62–76. https://doi.org/10.1037/a0013073
Glisson, C., Schoenwald, S. K., Hemmelgarn, A., Green, P., Dukes, D., Armstrong, K. S., & Chapman, J. E. (2010). Randomized trial of MST and ARC in a two-Level EBT implementation strategy. Journal of Consulting and Clinical Psychology, 78(4), 537–550. https://doi.org/10.1037/a0019160
Glisson, C., Williams, N. J., Hemmelgarn, A., Procter, E., & Green, P. (2016). Aligning organizational priorities with ARC to improve youth mental health service outcomes. Journal of Consulting and Clinical Psychology, 84(8), 713–725. https://doi.org/10.1037/ccp0000107
Harrison, D. A., & Klein, K. J. (2007). What’s the difference? Diversity constructs as separation, variety, or disparity in organizations. Academy of Management Review, 32(4), 1199–1228. https://doi.org/10.5465/amr.2007.26586096
Harvey, J.-F., Johnson, K. J., Roloff, K. S., & Edmondson, A. C. (2019). From orientation to behavior: The interplay between learning orientation, open-mindedness, and psychological safety in team learning. Human Relations, 72, 1726–1751. https://doi.org/10.1177/0018726718817812
Hasson, H., von Thiele Schwarz, U., Nielsen, K., & Tafvelin, S. (2016). Are we all in the same boat? The role of perceptual distance in organizational health interventions. Stress and Health, 32(4), 294–303. https://doi.org/10.1002/smi.2703
Hawley, K. M., Cook, J. R., & Jensen-Doss, A. (2009). Do noncontingent incentives increase survey response rates among mental health providers? A randomized trial comparison. Administration and Policy in Mental Health and Mental Health Services Research, 36(5), 343–348. https://doi.org/10.1007/s10488-009-0225-z
Herd, A. M., Shuck, B. M., & Githens, R. P. (2018). Strategic human resource development alignment from the employee’s perspective: Initial development and proposition testing of a measure. Performance Improvement Quarterly, 31(3), 269–291. https://doi.org/10.1002/piq.21266
Hoagwood, K. E., Olin, S. S., Horwitz, S., McKay, M., Cleek, A., Gleacher, A., Lewandowski, E., Nadeem, E., Acri, M., Chor, K. H. B., Kuppinger, A., Burton, G., Weiss, D., Frank, S., Finnerty, M., Bradbury, D. M., Woodlock, K. M., & Hogan, M. (2014). Scaling up evidence-based practices for children and families in New York state: Toward evidence-based policies on implementation for state mental health systems. Journal of Clinical Child & Adolescent Psychology, 43(2), 145–157. https://doi.org/10.1080/15374416.2013.869749
Jung, O. S., Kundu, P., Edmondson, A. C., Hegde, J., Agazaryan, N., Steinberg, M., & Raldow, A. (2021). Resilience vs. Vulnerability: Psychological safety and reporting of near misses with varying proximity to harm in radiation oncology. The Joint Commission Journal on Quality and Patient Safety, 47(1), 15–22. https://doi.org/10.1016/j.jcjq.2020.09.005
Lau, A. S., & Brookman-Frazee, L. (2016). The 4KEEPS study: Identifying predictors of sustainment of multiple practices fiscally mandated in children’s mental health services. Implementation Science, 11(1), 31. https://doi.org/10.1186/s13012-016-0388-4
Lau, A. S., Lind, T., Crawley, M., Rodriguez, A., Smith, A., & Brookman-Frazee, L. (2020). When do therapists stop using evidence-based practices? Findings from a mixed method study on system-driven implementation of multiple EBPs for children. Administration and Policy in Mental Health and Mental Health Services Research, 47(2), 323–337. https://doi.org/10.1007/s10488-019-00987-2
LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11(4), 815–852. https://doi.org/10.1177/1094428106296642
Lind, T., Lau, A. S., Cox, J. R., Chlebowski, C., Lui, J., Motamedi, M., & Brookman-Frazee, L. (2021, November 18). Predictors of therapist delivery of evidence-based strategies in a multiple EBP implementation context. Annual Convention of the Association of Behavioral and Cognitive Therapies, New Orleans, LA, USA.
Lind, T., Lau, A. S., Cox, J. R., Chlebowski, C., Lui, J., Motamedi, M., Byeon, Y. V., & Brookman-Frazee, L. (under review). Therapist delivery of evidence-based practice (EBP) strategies in a system-driven implementation of multiple EBPs for children.
London, M., & Smither, J. W. (1995). Can multi-source feedback change perceptions of goal accomplishment, self-evaluations, and performance-related outcomes? Theory-based applications and directions for research. Personnel Psychology, 48(4), 803–839. https://doi.org/10.1111/j.1744-6570.1995.tb01782.x
Los Angeles County Department of Mental Health. (n.d.). About DMH. Los Angeles County Department of Mental Health. https://dmh.lacounty.gov/about/
Lucid, L., Meza, R., Pullmann, M. D., Jungbluth, N., Deblinger, E., & Dorsey, S. (2018). Supervision in community mental health: Understanding intensity of EBT focus. Behavior Therapy, 49(4), 481–493. https://doi.org/10.1016/j.beth.2017.12.007
Marsick, V. J., & Watkins, K. E. (2003). Demonstrating the value of an organization’s learning culture: The dimensions of the learning organization questionnaire. Advances in Developing Human Resources, 5(2), 132–151. https://doi.org/10.1177/1523422303005002002
Monroe-DeVita, M., Morse, G., & Bond, G. R. (2012). Program fidelity and beyond: Multiple strategies and criteria for ensuring quality of assertive community treatment. Psychiatric Services., 63(8), 743–750. https://doi.org/10.1176/appi.ps.201100015
Morris, J. A., Brotheridge, C. M., & Urbanski, J. C. (2005). Bringing humility to leadership: Antecedents and consequences of leader humility. Human Relations, 58, 1323–1350. https://doi.org/10.1177/0018726705059929
Nakamura, B. J., Mueller, C. W., Higa-McMillan, C., Okamura, K. H., Chang, J. P., Slavin, L., & Shimabukuro, S. (2014). Engineering youth service system infrastructure: Hawaii’s continued efforts at large-scale implementation through knowledge management strategies. Journal of Clinical Child & Adolescent Psychology, 43(2), 179–189. https://doi.org/10.1080/15374416.2013.812039
Nembhard, I. M., & Edmondson, A. C. (2006). Making it safe: The effects of leader inclusiveness and professional status on psychological safety and improvement efforts in health care teams. Journal of Organizational Behavior, 27(7), 941–966. https://doi.org/10.1002/job.413
Newman, A., Donohue, R., & Eva, N. (2017). Psychological safety: A systematic review of the literature. Human Resource Management Review, 27(3), 521–535. https://doi.org/10.1016/j.hrmr.2017.01.001
O’Donovan, R., & McAuliffe, E. (2020a). A systematic review exploring the content and outcomes of interventions to improve psychological safety, speaking up and voice behaviour. BMC Health Services Research, 20(1), 101. https://doi.org/10.1186/s12913-020-4931-2
O’Donovan, R., & McAuliffe, E. (2020b). A systematic review of factors that enable psychological safety in healthcare teams. International Journal for Quality in Health Care, 32(4), 240–250. https://doi.org/10.1093/intqhc/mzaa025
Owens, B. P., & Hekman, D. R. (2012). Modeling how to grow: An inductive examination of humble leader behaviors, contingencies, and outcomes. Academy of Management Journal, 55(4), 787–818. https://doi.org/10.5465/amj.2010.0441
Powell, B. J., Mandell, D. S., Hadley, T. R., Rubin, R. M., Evans, A. C., Hurford, M. O., & Beidas, R. S. (2017). Are general and strategic measures of organizational context and leadership associated with knowledge and attitudes toward evidence-based practices in public behavioral health settings? A Cross-Sectional Observational Study. Implementation Science, 12(1), 64. https://doi.org/10.1186/s13012-017-0593-9
Roberson, Q. M., Sturman, M. C., & Simons, T. L. (2007). Does the measure of dispersion matter in multilevel research? A comparison of the relative performance of dispersion indexes. Organizational Research Methods, 10(4), 564–588. https://doi.org/10.1177/1094428106294746
Shanock, L. R., Baran, B. E., Gentry, W. A., Pattison, S. C., & Heggestad, E. D. (2010). Polynomial regression with response surface analysis: A powerful approach for examining moderation and overcoming limitations of difference scores. Journal of Business and Psychology, 25(4), 543–554.
Singer, S., Lin, S., Falwell, A., Gaba, D., & Baker, L. (2009). Relationship of safety climate and safety performance in hospitals. Health Services Research, 44(2p1), 399–421. https://doi.org/10.1111/j.1475-6773.2008.00918.x
Tafvelin, S., & Hasson, H. (2019). Agreement of safety climate: Does it affect employees’ and managers’ health and work performance? Journal of Occupational and Environmental Medicine, 61(4), e125. https://doi.org/10.1097/JOM.0000000000001542
Tafvelin, S., Nielsen, K., Abildgaard, J. S., Richter, A., von Thiele Schwarz, U., & Hasson, H. (2019). Leader-team perceptual distance affects outcomes of leadership training: Examining safety leadership and follower safety self-efficacy. Safety Science, 120, 25–31. https://doi.org/10.1016/j.ssci.2019.06.019
Tafvelin, S., von Thiele Schwarz, U., & Hasson, H. (2017). In agreement? Leader-team perceptual distance in organizational learning affects work performance. Journal of Business Research, 75, 1–7. https://doi.org/10.1016/j.jbusres.2017.01.016
Tsai, C.-Y., Kim, J., Jin, F., Jun, M., Cheong, M., & Yammarino, F. J. (2022). Polynomial regression analysis and response surface methodology in leadership research. The Leadership Quarterly, 33(1), 101592. https://doi.org/10.1016/j.leaqua.2021.101592
U.S. Census Bureau. (2019). B03002 Hispanic or Latino origin by race—Los Angeles County—2019 American Community Survey 1-Year Estimates.
Volk, S. C., & Zerfass, A. (2018). Alignment: Explicating a key concept in strategic communication. International Journal of Strategic Communication, 12(4), 433–451. https://doi.org/10.1080/1553118X.2018.1452742
von Thiele Schwarz, U., & Hasson, H. (2013). Alignment for achieving a healthy organization. In G. F. Bauer & G. J. Jenny (Eds.), Salutogenic organizations and change: The concepts behind organizational health intervention research (pp. 107–125). Springer.
Weiner, B. J., Belden, C. M., Bergmire, D. M., & Johnston, M. (2011). The meaning and measurement of implementation climate. Implementation Science, 6(1), 78. https://doi.org/10.1186/1748-5908-6-78
Whittington, J. L., Coker, R. H., Goodwin, V. L., Ickes, W., & Murray, B. (2009). Transactional leadership revisited: Self–other agreement and its consequences. Journal of Applied Social Psychology, 39(8), 1860–1886. https://doi.org/10.1111/j.1559-1816.2009.00507.x
Williams, N. J., & Beidas, R. S. (2019). Annual research review: The state of implementation science in child psychology and psychiatry: A review and suggestions to advance the field. Journal of Child Psychology and Psychiatry, 60(4), 430–450. https://doi.org/10.1111/jcpp.12960
Williams, N. J., Ehrhart, M. G., Aarons, G. A., Marcus, S. C., & Beidas, R. S. (2018). Linking molar organizational climate and strategic implementation climate to clinicians’ use of evidence-based psychotherapy techniques: Cross-sectional and lagged analyses from a 2-year observational study. Implementation Science, 13(1), 85. https://doi.org/10.1186/s13012-018-0781-2
Williams, N. J., & Glisson, C. (2020). Changing organizational social context to support evidence-based practice implementation: A conceptual and empirical review. In B. Albers, A. Shlonsky, & R. Mildon (Eds.), Implementation science 3.0 (pp. 145–172). Springer. https://doi.org/10.1007/978-3-030-03874-8_6
Wood, L., Hassell, A., Whitehouse, A., Bullock, A., & Wall, D. (2006). A literature review of multi-source feedback systems within and without health services, leading to 10 tips for their successful design. Medical Teacher, 28(7), e185–e191. https://doi.org/10.1080/01421590600834286
Funding
This work was supported by National Institute of Mental Health (NIMH) Grant R01 MH100134 to Drs. Lau and Brookman-Frazee.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation and data collection were performed by Drs. ASL and LB-F. Literature review and data analysis were performed by Ms. YVB. Analyses were interpreted by all authors. The first draft of the manuscript was written by Ms. YVB and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflicts of interest
Drs. Brookman-Frazee and Aarons are members of the editorial board for this journal and should be excluded from the peer review process. There are no other potential financial or non-financial conflicts of interest for any of the authors.
Ethical Approval
All procedures for this study were reviewed and approved by the Institutional Review Boards at Los Angeles County Department of Mental Health, University of California, Los Angeles, and University of California, San Diego.
Research Involving Human and Animal Participants
This research involved self-report data from human subjects collected via an online survey. All procedures performed in this study involving human participants were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. The study was granted a waiver of signed informed consent and a written information sheet, approved by the UCLA IRB, was provided to each participant on the first page of the online survey.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Byeon, Y.V., Brookman-Frazee, L., Aarons, G.A. et al. Misalignment in Community Mental Health Leader and Therapist Ratings of Psychological Safety Climate Predicts Therapist Self-Efficacy with Evidence-Based Practices (EBPs). Adm Policy Ment Health 50, 673–684 (2023). https://doi.org/10.1007/s10488-023-01269-8
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10488-023-01269-8