Evaluating the before operational stress program: comparing in-person and virtual delivery

Introduction Public safety personnel (PSP) are at increased risk for posttraumatic stress injuries (PTSI). Before Operational Stress (BOS) is a mental health program for PSP with preliminary support mitigating PTSI. The current study compared the effectiveness of delivering BOS in-person by a registered clinician (i.e., Intensive) to virtually delivery by a trained clinician (i.e., Classroom). Methods Canadian PSP completed the Intensive (n = 118; 61.9% male) or Classroom (n = 149; 50.3% male) program, with self-report surveys at pre-, post-, 1 month, and 4 months follow-ups. Results Multilevel modelling evidenced comparable reductions in anxiety (p < 0.05, ES = 0.21) and emotional regulation difficulties (ps < 0.05, ESs = 0.20, 0.25) over time with no significant difference between modalities. Participants discussed benefits of the delivery modality they received. Discussion The results support virtual delivery of the BOS program (Classroom) as an accessible mental health training option for PSP, producing effects comparable to in-person delivery by clinicians.


Introduction
Canadians rely on diverse public safety personnel (PSP; e.g., police, firefighters, paramedics) to ensure their safety and well-being (Mendicino and Blair, 2022).PSP are at increased risk for posttraumatic stress injuries (PTSI) (Carleton et al., 2020a;Heber et al., 2023) resulting from operational and organizational stressors (Carleton et al., 2020a).Almost half (44.5%) of PSP screen positive for one or more mental health disorders (Vig et al., 2020) and many report lifetime suicidal ideation (27.8%), planning (13.3%), and attempts (4.6%) (Carleton et al., 2018a).PTSI are associated with frequent exposures to potentially psychologically traumatic events (PPTEs; i.e., direct or indirect exposures to actual or Ioachim et al. 10.3389/fpsyg.2024.1382614Frontiers in Psychology 02 frontiersin.orgthreatened death, serious injury, or sexual violence) (Carleton et al., 2019a;Heber et al., 2023).PTSI are a substantial concern for PSP mental health and the Canadian government has recently developed an action plan to address challenges related to these mental health difficulties (Public Safety Canada, 2019).There are several effective treatments for PTSI, such as cognitive behavioural therapy (CBT), that appear beneficial for PSP (Foa and Rothbaum, 2001;Ponniah and Hollon, 2009;Carleton, 2021;Hadjistavropoulos et al., 2021).Proactive solutions to mitigate PTSI also exist, often focusing on providing training or peer support to bolster resilience, minimize stigma, and develop individual stress management skills (Carleton et al., 2020b;Stelnicki et al., 2021).There has been relatively little research on such proactive efforts, with the available evidence suggesting small time-limited effects, necessitating recommendations for larger longitudinal research efforts (Anderson et al., 2020;Di Nota et al., 2022).

Overview of the before operational stress program
The Before Operational Stress (BOS) program was designed to provide access to effective evidence-based mental health training for PSP (McElheran et al., 2020;Stelnicki et al., 2021).The BOS programming is based on teaching core CBT skills and providing evidence informed learning content, taught by a clinician trained in CBT principles and familiar with PSP culture and treatment.BOS program combines theoretical and experiential learning procedures designed to improve resiliency, strengthen interpersonal relationships, and mitigate the effects of operational stressors.Participants complete one module per week for 8 weeks.The first six modules involve teaching participants to identify, understand, and navigate the connection between thoughts, emotions, physiological sensations, and behaviour.The final two modules focus on enhancing interpersonal relationships by teaching communication skills and empathy.Throughout the program, participants explore various mental health related topics to expand their knowledge of OSI and effective coping strategies.The program also emphasizes the importance of PSP maintaining healthy, authentic relationships, which can provide important mental health benefits for PSP as previous research suggests they are more likely to seek support from spouses and friends than professionals (Carleton et al., 2019b(Carleton et al., , 2020b;;Nisbet et al., 2023).
The BOS Intensive program is facilitated in-person by a trained clinician in a group setting, following the one module per week for eight weeks model.Each 2 hour module-session is approximately evenly divided between a facilitator-led didactic component communicating program content, and 1 hour of group processing where participants share thoughts on the program content and discuss its application in their lives.A prior evaluation of the BOS Intensive program evidenced small but statistically significant improvements in PTSD symptoms and quality of life measures, increases in perceived social support, and reductions in mental health stigma associated with the training (Stelnicki et al., 2021).This evaluation was conducted with a smaller sample of participants who completed the training through the Original BOS Intensive (in-person) delivery modality and surveyed participants at four time points (before and after the training, and at 1 month and 3 month follow-ups).The available outcomes were associated with improved communication skills and more positive behaviours toward family members, as evidenced by qualitative responses.
The BOS Classroom program, consisting of 1 h sessions delivered virtually by a trained clinician, was developed to enhance training accessibility for individuals in areas lacking BOS-trained clinicians.The BOS Classroom modality removes the group processing aspect of the Intensive training to solely focus on delivery of the didactic component, therein shortening each module-session to approximately one hour.BOS Classroom became particularly important for maintaining training accessibility during public gathering restrictions imposed during the COVID-19 pandemic.The virtual training also helped address logistical access barriers by providing flexible timing and avoiding stigma barriers still prevalent in PSP workplaces (Rice et al., 2019;Hadjistavropoulos et al., 2021).
To date there has been no evaluation of the effectiveness of the BOS Classroom program and, by extension, no comparative assessment with the BOS Intensive program.Extant research indicates internet-based Cognitive Behavioural Therapy (ICBT) can be as effective as face-to-face CBT in treating a range of mental health challenges, offering similar therapeutic benefits, but with the added convenience of accessibility and flexibility (Andersson et al., 2019;Hadjistavropoulos et al., 2021;Thew et al., 2022).Previous evaluations of mindfulness training with military personnel demonstrated in-person training as producing better improvements in sleepiness, pain, and energy, than virtual training (Rice and Schroeder, 2021).The researchers suggested the in-person training benefits were supported by having longer sessions, stronger interpersonal bonds between the instructor and classmates (Rice and Schroeder, 2021), face-to-face interactions, greater accountability, and in-class participation (Rice et al., 2019).

The current study
The current study was designed to evaluate the effectiveness of the BOS Classroom program for improving mental health.The study hypotheses were: (1) participation in the BOS program would be associated with reductions in mental health symptoms, substance use, and mental health stigma, as well as increases in perceived social support, emotional regulation, resilience, and quality of life; and (2) the outcomes of the BOS Classroom program would be largely comparable to the BOS Intensive program.
We anticipate that the effect sizes in the BOS Classroom will be smaller than those in the BOS Intensive, although the direction of the effects is expected to be consistent across both modalities.This difference in effect sizes is expected due to the distinct characteristics of the Classroom modality, such as reduced group processing and shorter session durations.Although previous research suggests in-personal and virtual CBT training and treatment programs can have similar effectiveness (Spek et al., 2007;Andersson et al., 2016;Carlbring et al., 2018;Rice et al., 2018;Zhang et al., 2022;Alavi et al., 2023), BOS Classroom also features a reduced session time and omits the group processing component, which may affect the overall effectiveness of the training.The current study builds upon the findings of the previous independent evaluation of the BOS program (Stelnicki et al., 2021).Our study extends this work by comparing delivery modalities and employing a comprehensive mixed-methods approach, with a specific focus on contrasting the effectiveness of in-person and virtual delivery modalities.

Procedure
The current study was approved by the University of Regina Institutional Research Ethics board .Data were collected for two training modalities: BOS Intensive (i.e., in-person with group processing components) and BOS Classroom (i.e., virtual, didactic components only).The quasi-experimental design resulted from convenience samples arising from BOS Intensive groups being scheduled in-person until implementation of COVID-19 pandemic gathering restrictions, with BOS Classroom groups being scheduled after onset of the gathering restrictions.All participants completed a standardized intake interview prior to starting the BOS program, during which they were informed about potential voluntary participation in an independent research study evaluating the BOS training.Participants were screened prior to training for acute mental health distress or severe PTSD symptoms, and those in need were referred to therapy intervention resources.Participants who expressed interest were emailed a consent form along with a link to the first survey.Due to confidentiality assured during BOS training, data was only recorded from participants who expressed interest in the study, and the total number of participants in the BOS training at the time outside of study participation is not available.The research participation was voluntary, anonymous, and did not impact eligibility for the BOS program.Neither the clinical facilitators nor other group members were aware which members chose to participate in the study.No questions were mandatory, and participants could withdraw at any time without consequences for their training.Surveys were administered at four time points: pre-training, post-training, 1 month follow-up, and 4 months follow-up.All surveys contained the same measures, with pre-training also including sociodemographic measures, and post-training having an additional five open-ended questions requesting feedback on participant experiences in the training program.Although training recruitment was aimed at early career recruits, training was open to all career stages.As PSP will likely experience multiple exposure to PPTEs in their careers, BOS is intended as a proactive measure for any future exposures a PSP may experience.

Measures
Study measures were chosen based on module content of the BOS program and to facilitate comparison with the BOS pilot study evaluation (Stelnicki et al., 2021).

Alcohol use disorders identification test
The Alcohol Use Disorders Identification Test [AUDIT; Babor et al., 2001] is a 10-item self-report measure of potentially hazardous alcohol use.Participants are asked to rate each item on a 5-point Likert scale ranging from 0 (never) to 4 (daily or almost daily).Higher scores indicate more potentially hazardous alcohol use.The AUDIT is widely employed and has evidence of adequate psychometric properties (Reinert and Allen, 2002;de Meneses-Gaya et al., 2009;Peng et al., 2012).AUDIT scores can be used to screen for hazardous alcohol use (>7) and alcohol dependence (>15).

Brief resiliency scale
The Brief Resiliency Scale (BRS; Smith et al., 2008) is a 6-item self-report measure of resilience.Participants rate each item on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree).Items are summed to produce a total score, with higher scores indicating higher resiliency.The BRS has evidence of adequate internal consistency and test-retest reliability (Windle et al., 2011).

Depression, anxiety, and stress scale
The Depression, Anxiety, and Stress Scale [DASS-21;Lovibond and Lovibond, 1995] is a 21-item self-report measure of designed to measure the negative emotional states of depression, anxiety, and stress.Each subscale consists of 7 items scored on a 5-point Likert scale ranging from 0 (did not apply to me at all) to 4 (applied to me very much or most of the time).Higher scores indicate greater symptom levels.The DASS-21 has evidence of adequate internal consistency, construct validity, and convergent and discriminant validity (Henry and Crawford, 2005).The DASS-21 subscale scores can be used to screen for clinicallysignificant depression (>20), anxiety (>14), and stress (>25).

Difficulties in emotional regulation scale
The Difficulties in Emotional Regulation Scale (DERS; Gratz and Roemer, 2004) is a 36-item self-report measure of difficulties with emotional regulation.Participants rate each item on a 5-point Likert scale ranging from 1 (almost never) to 5 (almost always).All items can be summed to produce a total score, with higher scores indicating a greater degree of emotional dysregulation.Item subsets can be summed to produce subscales related to emotional responses (i.e., nonacceptance of emotional responses, lack of emotional awareness, limited access to emotional regulation strategies, and limited emotional clarity).The DERS has evidence of adequate reliability, as well as construct and concurrent validity (Gratz and Roemer, 2004;Bardeen et al., 2012;Fowler et al., 2014;Hallion et al., 2018), and may help predict responses to cognitive-behavioural therapy (Hallion et al., 2018(Hallion et al., ). 10.3389/fpsyg.2024.1382614 .1382614Frontiers in Psychology 04 frontiersin.org

Opening minds survey for workplace attitudes
The Opening Minds Survey for Workplace Attitudes (OMSWA; Szeto et al., 2013) is an 11-item self-report measure of attitudes towards people with mental illness.Participants rate each item on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree).Items are summed to produce a total score with higher scores indicating a higher degree of stigma in the workplace.The OMSWA has been widely employed by the Mental Health Commission of Canada to measure stigma (Krakauer et al., 2020) and with evidence of adequate factor validity for the 9-item version (Boehme et al., 2022).

PTSD checklist for DSM-5
The PTSD Checklist for DSM-5 (PCL-5; Weathers et al., 2013) PCL-5 is a 20-item self-report measure used to assess PTSD symptoms in the past month.Participants who identify exposure to at least one PPTE are then asked to select which exposure has caused them the most difficulty recently and answer questions regarding how much they have been bothered by different aspects of that event over the past month.Participants rate each item on a 5-point Likert scale ranging from 0 (not at all) to 4 (extremely).All item scores can be summed to produce a total score, and subsets of items can be summed to produce subscale scores reflecting DSM-5 symptom clusters B (re-experiencing/re-living), C (avoiding reminders of the incident), D (negative thoughts and mood), and E (hyper arousal/alertness).Higher scores indicate greater symptom levels.The PCL-5 has evidence of adequate reliability, as well as structural, convergent, and discriminant validity (Blevins et al., 2015;Ashbaugh et al., 2016).A positive screen for probable PTSD can be made for participants who endorse symptoms in each PTSD cluster and exceed the minimum clinical cutoff of 32 for the total PCL-5 score.

Social provisions scale
The Social Provisions Scale (SPS10; Cutrona and Russell, 1987) is a 10-item self-report measure of extent of social support.Participants rate each item on a 4-point Likert-type scale ranging from 1 (strongly disagree) to 4 (strongly agree).All items can be summed to produce a total score, and the SPS10 has evidence of adequate reliability and convergent validity (Gottlieb and Bergen, 2010).

WHO quality of life-BREF (WHOQOL-BREF)
The WHO Quality of Life-BREF (WHOQOL-BREF; WHOQOL Group, 1998) is a 25-item self-report measure of quality of life.Participants rate each item on a 5-point Likert scale ranging from 1 (not at all) to 5 (an extreme amount).The mean of all items is calculated to produce a total score, with item subset means calculated to produce subscale scores describing each of four domains: physical health, psychological health, social relationships, and environmental quality of life.We analyzed domains 2, 3, and 4, which relate to psychological and social quality of life.Higher mean scores indicate greater quality of life.The scale has evidence of adequate psychometric properties (Skevington et al., 2004).

Qualitative data and analyses
Participants were asked five open-ended questions at the end of the post-training survey (i.e., upon BOS program completion): (1) What has been the most helpful aspect of BOS for you?; (2) What has been the least helpful aspect of BOS for you?; (3) Has anything gotten better for you as a result of BOS?If so, please describe; (4) Has anything gotten worse for you as a result of BOS?If so, please describe; and (5) Please use the space below to provide any other comments you would like about your participation in BOS.Of the initial participant responses, 9 (n = 9) were excluded due to missing data, resulting in 30 Intensive and 45 Classroom responders, totaling 75 (n = 75) participants.Responses were anonymized and analyzed in NVivo 12. Two authors (MR and AW) used the coding procedures outlined in Miles et al. (2018) to separately inductively analyze the data.The same authors then collaborated to create broad themes by comparing their two code lists, producing a master coding framework, and recoding the data.A comparison query was run to determine the degree of overlap between coders (calculated at 80% agreement with a Cohen's Kappa of 0.40, suggesting a fair level of agreement beyond what might be expected from chance) and to reach consensus on codes of disagreement (i.e., what theme best captured codes of discrepancy between authors).The authors then re-visited the coded data and addressed discrepancies in themes to achieve 100% agreement and co-constructed definitions to closely represent participant experiences by discussing the coded themes and the associated interrelationships.A matrix coding query was used to analyze the differences in theme references between Intensive and Classroom modalities and to clarify final participant counts for each theme.

Quantitative analyses
Descriptive analyses provided information about the sociodemographic characteristics of participants in the Intensive and Classroom modalities (Table 1).Descriptive analyses characterized mental health and resilience scale variables over time (Table 2).Generalized Estimating Equations were used to assess whether the number of participants screening positive for a particular mental health challenge changed statistically significantly over time (Table 3) (Andreski et al., 1998;Heine et al., 2011).Modality was included as a factor to assess whether the number of participants with a positive screen differed between Intensive and Classroom modalities.Interaction effects were initially tested, but none were statistically significant (all ps > 0.05), so none were retained for the final model.
A multi-level modelling (MLM) approach was employed to assess for changes in mean self-report scores over time, and to assess whether any changes differed across delivery modalities (i.e., Intensive or Classroom, Table 4).The MLM approach provides accurate estimates when some individuals have missing timepoints (Heck et al., 2013;West et al., 2015), and also allowed for more close comparison to the previous BOS Intensive evaluation.Model fits were estimated using restricted maximum likelihood (REML) via the MIXED command in SPSS Version 28.The MLM strategy for each scale was as follows: analyses began with a two-level model consisting of a fixed effect for Time, a random intercept and slope for Time at the individual level, and the first order autocorrelation structure for the within individual covariance matrix at the repeated measures level.The random slope for Time was removed from the model due to non-convergence, which is consistent with previous BOS program evaluations using the same model criteria (Stelnicki et al., 2021).Modality was then included as a moderator variable by adding fixed effects for Modality   intraclass correlation coefficients included for interpretation.Standardized effect sizes (ES) and intraclass correlation coefficients (ICC) for the MLM analysis were calculated manually using estimates of fixed and random effects.Standardized effect sizes were calculated for each fixed effect using a pooled variance estimate from the final model for each scale.Using pooled variance to standardize effect size estimates accounts for the within and between individual variance in repeated measures designs (Pustejovsky et al., 2014;Westfall et al., 2014).The standardized effect sizes can be interpreted like Cohen's d (Pustejovsky et al., 2014), in which 0.20 represents a small effect, 0.50 represents a medium effect, and 0.80 represents a large effect (Pleil et al., 2018).ICCs were calculated for each applicable scale or subscale using the between individual (random intercept) variance estimate and the residual variance estimate from the empty random intercept model.The empty random intercept model consisted of a random intercept at the individual level, and no fixed effects, estimated with REML.In the absence of model predictors, the ICC provides an estimate of the proportion of variance attributable to individual differences (Raudenbush and Bryk, 2002;Hox and De Leeuw, 2003).

Results
Sociodemographic characteristics of participants in both modalities were largely comparable (Table 1).The only statistically significant effect of modality for positive screens indicated Classroom modality participants at pre-training had a lower prevalence of positive screens for potentially hazardous alcohol use than Intensive modality participants (Table 3).The differences in prevalence of positive screens for mental health disorders between the modalities were largely attributable to individual differences between participants (Table 3).
There were no statistically significant differences between modalities for other measures, except that OMSWA total scores were statistically significantly higher (p < 0.05) among Intensive participants (Table 4).Average change estimates (Table 4) indicated PCL-5 Cluster E (hyper arousal/alertness) scores increased statistically significantly (p < 0.05) for all participants from pre-to post-training.Anxiety and environmental quality of life subscale scores decreased (p < 0.05) for all participants from pre-training to 1-month follow-up.Emotional regulation total scores decreased from pre-training to 1 month follow-up (p < 0.05), and pre-training to 4-month follow-up (p < 0.05) (Table 4).Emotional regulation subscale scores decreased (p < 0.05) for "Non Acceptance of Emotional Responses" and "Emotional Awareness" from pre-training to 4 month follow-up."Emotional Clarity" subscale scores evidenced a statistically significant increase from pre-to post-training, and a statistically significant decrease from pre-training to 1 month follow-up.Decreases in resilience from pre-to post-training, and from pre-training to 4 month follow-up, and increases in resilience from pre-training to 1 month follow-up were not statistically significant.
All model random effects were statistically significant (ps < 0.05), indicating a large proportion of the variance observed in outcome measures across time was attributable to initial differences between individuals, as well as differences within each individual.The results

Qualitative analyses results
The most heavily emphasized theme across participant responses was insight and awareness changes, with many participants describing thinking about themselves and their experiences in new ways (cognitive); behaving in different ways or acting upon new awareness (behavioural); and understanding their feelings and emotional responses, including being more attuned to bodily sensations (emotional).Participants in both modalities described increased awareness or insight within cognitive (Intensive: n = 11; Classroom: n = 24), emotional (Intensive: n = 13; Classroom: n = 6), and behavioural domains (Intensive: n = 9; Classroom: n = 5).Intensive participants more frequently described increased emotional and behavioural awareness, whereas Classroom participants more frequently described increased cognitive awareness.For example, one Intensive participant described increased attention to and recognition of their emotional experiences: "I have been able to recognize certain emotions as they are occurring and work through them, opposed to before when I would just become angry because I did not understand emotion" (Participant 17).Intensive (n = 5) and Classroom (n = 2) participants also described changes in their familial relationships: "I've started communicating better with my wife and being able to relate and explain some of her feelings and thoughts to what I've learned in the program as well" (Intensive Participant 15).
The stigma reduction theme included participant descriptions of feeling less different from others or feeling less alone with their mental health symptoms, either via the explicit learning content (i.e., learning about brain-based or physiological responses to PPTE or other stressful events) or via group discussion of said content (i.e., relating to others' experiences, experiencing normalization, and validation of responses to trauma or stress by sharing and hearing others' stories).Intensive (n = 14) and Classroom (n = 4) modality participants described reductions in stigma via the group dynamic: "I have found a small cadre of others in the same boat as me, people    I can relate with who had also stepped forward to say 'I'm broken'" (Intensive Participant 4).Classroom (n = 7) and Intensive (n = 2) modality participants described the learning content contributed to reductions in feelings of isolation: "The program has been very helpful, but the one thing that really was … [that] PTSD is not a mental illness but a brain injury." (Classroom Participant 39).This variation indicates that while both modalities were perceived as beneficial, consistent with the program content emphasized in each modality, Intensive participants more frequently cited group discussions and Classroom participants discussed the learning content.
The delivery theme included participant descriptions of program accessibility, quality, and delivery of the program.Classroom modality participants (n = 17) expressed negativity towards the delivery method, indicating a preference for Intensive delivery.There were also Classroom modality participants (n = 18) who reported feeling they had missed out on the benefits of group sessions and described online participation as difficult: "… very hard to express your feelings and receive feedback… The program would be far more beneficial if it was in a group setting, face to face with others" (Classroom participant 52).Intensive modality participants (n = 8) more often described participation challenges such as scheduling of the sessions and time required to attend compared to Classroom participants (n = 2).

Discussion
The current study was designed to evaluate the effectiveness of the virtual modality of the BOS program (BOS Classroom) as compared to the existing in-person delivery (BOS Intensive).The current results support the Classroom modality as generally comparable to the Intensive modality in terms of changes in selfreported mental health symptom outcomes.Across both modalities, the BOS program was associated with small, but statistically significant, improvements in self-reported anxiety and emotional regulation.The current results are consistent with a previous evaluation of BOS Intensive (Stelnicki et al., 2021), but incrementally evidence some of the small, statistically significant improvements in anxiety and emotional regulation, some of which were sustained 4 months after completing the training.
Consistent with our hypotheses, outcomes following the BOS program were largely comparable across both Classroom and Intensive modalities, with no differences in effects observed between modalities.The only modality effect was for the OMSWA measure, which differed statistically significantly between groups at initial scores, indicating changes in outcome measures did not meaningfully differ between delivery types.The proportion of participants who screened positive for mental health disorders were also largely comparable across modalities, except for a higher prevalence of potentially hazardous alcohol use positive screens among Intensive participants at pre-training.There were no statistically significant decreases in the number of participants who screened positive for mental health disorders over time after controlling for individual differences, suggesting any changes reflected dynamic individual circumstances, differences in relative engagement with the BOS material, or other unmeasured variables.The results were consistent with previous research evidencing Mindfulness-based Stress Reduction training as comparable across virtual and in-person formats among US military personnel and Veterans (Rice et al., 2018(Rice et al., , 2019)).
There were small, statistically significant improvements in anxiety symptoms and difficulties with emotional regulation across both modalities, which were sustained 4 months after the training.Opentext responses corroborate and contextualize the quantitative results, with participants from both modalities reporting new insights into their own thoughts and emotions concerning stressful situations.There were no statistically significant reductions in mental health stigma and feelings of isolation for participants in either modality, but participants who provided qualitative feedback described stigma reductions as important elements of the BOS program experience.Classroom modality participants primarily attributed reductions in self-stigma to the didactic component of BOS, whereas Intensive modality participants attributed the same reductions to group sharing.The feedback is consistent with prior research regarding the perceived potential impact of psychoeducation (Ricciardelli et al., 2020) and social validation (Kosmicki and Glickauf-Hughes, 1997;Cox et al., 2017;Yalom and Leszcz, 2020) for reducing stigmatizing beliefs.Reports of reduced stigma may be particularly important for PSP populations, as previous research has identified stigma towards mental health as a primary barrier to help-seeking behaviors in PSP (Newell et al., 2022).
Open-text responses also indicated that Intensive participants valued the group discussions, whereas Classroom modality participants described technical difficulties with online discussions sufficient to describe the discussions as detrimental to the overall training.The results contrast previous evidence suggesting preferences for virtual discussions perceived as less anxiety-provoking and more accessible (Rochlen et al., 2004;Fortier et al., 2022).Despite also facing some accessibility barriers while attending the training, participants in the BOS Intensive modality collectively expressed a preference for in-person groups.The individual variability in results and preferences, juxtaposed with the comparable results across modalities, suggests PSP may be best served by self-selecting modality training options that meet their current needs and preferences.
There was evidence of statistically significant increases in hyperarousal/alertness and decreases in environmental quality of life from pre-to post-training, possibly due to increased PPTE exposures concurrent with COVID-19 pandemic onset (Heber et al., 2020).Participants reported that the BOS program increased their mental health self-awareness, which may have facilitated increased attention to, and reporting of, the progressive pandemic impacts.In general populations, 40 to 70% of those who met screening criteria for PTSD symptoms no longer screened positive after CBT treatments or interventions (Bradley et al., 2005).In military and Veteran populations, many participants still report residual symptoms after CBT and prolonged exposure therapy (Bradley et al., 2005;Steenkamp et al., 2015;Allan et al., 2017), which may indicate these particular symptoms are resistant to treatment or are more successfully addressed over longer periods of time.Hyperarousal symptoms in particular were reported to be treatment resistant to typical CBT in military and Veteran populations (Crawford et al., 2019;Schnurr and Lunney, 2019;Miles et al., 2023).Promising alternative solutions have been explored in Veteran populations, such as mediation-based intervention (Crawford et al., 2019) to address this category of symptoms.Consistent with previous research results for BOS (Stelnicki et al., 2021) (Carleton et al., 2018b(Carleton et al., , 2019a;;Anderson et al., 2020).The BOS program presents a promising option for PSP mental health training as a function of small, but statistically significant, improvements in measures of anxiety, emotional regulation, and mental health stigma, which appear comparable across in-person and virtual modalities.Participants in the BOS Intensive program reported challenges with accessibility, but still reported a preference for in-person engagement.In addition, the virtual BOS training may be a viable alternative for PSP who find in-person sessions less accessible or convenient.Allowing PSP to select the modality that best aligns with their current circumstances and learning preferences could lead to increased engagement and, potentially, more effective training outcomes.

Limitations and future directions
The current study has several limitations that can help inform future research directions.First, the Classroom modality was introduced in response to growing accessibility needs and COVID-19 pandemic safety measures, which meant that the current data were based on a quasi-experimental design without randomized assignment to each modality.Second, there is no way to differentiate the relative impact of the COVID-19 pandemic from the impact of the BOS program using data from the current sample (Yu et al., 2020;Combden et al., 2022;Bouza et al., 2023) with PSP and healthcare workers appearing to have been disproportionately impacted by the COVID-19 pandemic (Heber et al., 2020;Marchildon et al., 2020;Hossain and Clatty, 2021;Cadell et al., 2022;Xue et al., 2022;Patel et al., 2023).The comparable effects found in the two modalities despite this time delay suggest the effects associated with BOS may be sustained even in highstress social environments.Third, there was considerable intrapersonal variability in many outcome measures, reducing the statistical power to detect additional small effects at 1 month and 4 month follow-up assessments.Future researchers should consider using explicit random assignment or participant self-selection and including measurements of skill acquisition and use as part of evaluating the relative impact of the BOS program.Fourth, there was substantial attrition during the study such that the sample size at the 4 month follow-up was modest and the associated results may reflect important self-selection biases.Fifth, the current qualitative evidence suggests important elements of participant experiences may not be entirely captured in quantitative surveys, highlighting the importance of nuanced assessment of participant perceptions of the program in program evaluations.

Conclusion
The current study evaluated BOS program delivery and expanded previous research on the BOS program by including a longer follow-up period, comparing the Intensive (i.e., in-person) and Classroom (i.e., virtual) delivery modalities, and including qualitative analyses of participant experiences.The current results were consistent with previous research and evidenced small, but statistically significant, improvements in anxiety and emotional regulation, some of which were sustained 4 months after training.Changes in mental health symptoms were largely comparable across Intensive and Classroom modalities; however, many participants reported a preference for the Intensive program despite acknowledging accessibility benefits of the Classroom modality.Participants reported perceiving stigma reductions as part of qualitative data collection that were not reflected in quantitative selfreport analyses.The comparable results across modalities suggests PSP may be best served by self-selecting modality training options that meet their current needs and preferences.

TABLE 1
Public safety personnel demographics by delivery modality.

TABLE 2
Self-report mental health measure metrics by time.

TABLE 2 (
Continued) Total percentages may not sum to 100 due to non-response or responding "other." PCL-5, posttraumatic stress disorder checklist for DSM-5 (Subscale cluster B-re-experiencing, cluster Cavoiding, cluster D, negative thoughts, cluster E-arousal/alertness); DASS, depression, anxiety, and stress scale; DERS, difficulties in emotion regulation scale (subscales for non-acceptance of emotional responses, lack of emotional awareness, limited access to emotional regulation strategies, lack of emotional clarity); WHOQOL, World Health Organization Quality of Life (Subscales for domain 2-psychological health, domain 3-social relationships, domain 4-environmental); BRS, brief resilience scale; SPS-10, social provisions scale; OMSWA, opening minds survey for workplace attitudes.

TABLE 3
Valid generalized estimating equations for psychological disorder criteria screeners by modality and timepoint, using intensive modality at T1 as baseline.

TABLE 4
Two-level MLM results for overall scale totals.
, standard error; CI, 95% confidence interval; ICC, intraclass correlation; PCL-5, posttraumatic stress disorder checklist for DSM-5 (subscale cluster B-re-experiencing, cluster Cavoiding, cluster D, negative thoughts, cluster E-arousal/alertness); DASS, depression, anxiety, and stress scale; DERS, difficulties in emotion regulation scale (subscales for non-acceptance of emotional responses, lack of emotional awareness, limited access to emotional regulation strategies, lack of emotional clarity); WHOQOL, World Health Organization Quality of Life (Subscales for domain 2-psychological health, domain 3-social relationships, domain 4-environmental); BRS, brief resilience scale; SPS-10, social provisions scale; OMSWA, opening minds survey for workplace attitudes.*p < 0.05, **p < 0.01, and ***p < 0.001.The AUDIT is not reported due to insufficient variance in the measure to estimate the model. SE there were no statistically significant 10.3389/fpsyg.2024.1382614Frontiers in Psychology 11 frontiersin.orgchanges in self-reported resilience among participants in either modality.Existing time-limited training interventions for PSP have generally demonstrated limited effectiveness for improving mental health outcomes