This study was preregistered on clinicaltrials.gov (identifier: NCT05956678) and received approval from the Committee for the Protection of Human Subjects at the University of California, Berkeley. Any protocol changes will be submitted or reported to clinicaltrials.gov, the Committee for the Protection of Human Subjects, the participating CMHCs, and in appropriate publications. If there are too many findings to reasonably interpret in one paper, findings may be separated into two or more papers. The present protocol used the SPIRIT reporting guidelines (see SPIRIT checklist in supplemental documents and Fig. 1) [35].
Participants
Participants in the Sustainment Phase are providers from CMHCs included in the Implementation or Train-the-Trainer Phases, for which the inclusion criteria were: 1) provision of publicly-funded adult mental health outpatient services and 2) support from CMHC leadership.
The inclusion criteria for providers in the present study are: 1) employed, able to deliver, or have delivered patient-facing services to patients within a CMHC1; 2) have attended a TranS-C training; 3) CMHC site of employment has been in the Sustainment Phase for at least three months; and 4) volunteer to participate and formally consent to participate. Note that CMHCs preferred to determine which providers were eligible to receive TranS-C training at each site (e.g., case managers, nurses, psychiatrists), because this aligns with their real-world practice.
It may be helpful to note that in the Implementation and Train-the-Trainer Phases, providers were trained to deliver TranS-C to adult patients who met criteria for SMI and exhibited a sleep or circadian problem. However, given the focus on provider-level data in the Sustainment Phase as well as feasibility and resource constraints, patient data are not assessed in the present study (see also Discussion).
Recruitment
Community Mental Health Centers
Building the CMHC network that forms the basis for this study began in August 2013 with outreach by the Principal Investigator of the three-phase trial (AGH). Originally, eight counties—generally consisting of three to 10 CMHC sites—agreed to participate in the Implementation Phase. At various stages of the study, recruitment of new counties and new CMHC sites continued in order to maximize provider and patient sample size goals for the Implementation and Train-the-Trainer Phases. For instance, all counties who participated in the Implementation Phase were invited to participate in the Train-the-Trainer Phase. Most elected to continue participating in the Train-the-Trainer Phase, but to account for fluctuations in engagement, another county (Lake County) was recruited for the Train-the-Trainer Phase. In general, sites were selected based on interest from partners and to balance diversity (e.g., race/ethnicity of patients served; urban vs. rural locations) with feasibility (e.g., manageable driving distance from University of California Berkeley). Sites in the following ten counties in California, United States are included in the Sustainment Phase recruitment efforts: Alameda, Contra Costa, Kings, Lake, Monterey, Placer, Santa Barbara, Santa Cruz, Solano, and Santa Clara. Note that sites in San Lois Obispo are also participating but are operating as part of Monterey County. A list of the study sites can be found on clinicaltrials.gov (NCT05956678).
Providers
For the Sustainment Phase, eligible providers are contacted by email and invited to complete a survey and interview about TranS-C. Providers are compensated for their time depending on local policies for receiving payment for research-related activities (e.g., compensated with gift card or treatment-related book).
Interventions
As mentioned above, two variations of TranS-C are tested in this trial: Standard TranS-C and Adapted TranS-C (see Table 1 for comparison). Both are delivered alongside the usual care offered by each CMHC. In the CMHCs, usual care consists of working with a service provider (e.g., psychologist, case manager, occupational therapist, psychiatrist, nurse practitioner) who provides direct mental health support from within their scope of practice. The patient might also be referred by that provider for other services as needed (e.g., healthcare, housing support, nutrition, vocational specialists, or peer advocacy). Occasionally patients receive treatment from interdisciplinary or residential teams, meaning their services are coordinated across multiple service providers. The TranS-C treatment conditions, along with the adaptation process for Adapted TranS-C, are described below. The modules that make up Standard and Adapted TranS-C are listed in Fig. 1 and described in detail elsewhere [33]. While the ordering of modules is broadly suggestive of the order of completion, providers are trained to be sensitive to the differences between patients as to which processes are key to maintaining their distress and to address these processes at an earlier stage of treatment. Although most providers deliver TranS-C via individual sessions, some choose to deliver it in a group setting. Of note, TranS-C was originally developed in English, then translated into Spanish and offered by Spanish-speaking providers during the Implementation Phase to expand access.
Table 1
Comparison of Standard and Adapted TranS-C Trainings, Sessions, and Modules
| Standard TranS-C | Adapted TranS-C |
Training | 6–8 hours | 4 hours |
Session Length | 50 minutes | 20 minutes |
Number of Sessions | 8 | 4 |
TranS-C Modules | | |
Cross-Cutting | | |
Functional Analysis | Yes | Yes |
Education | Yes | Yes |
Motivational Enhancement | Yes | Yes |
Goal Setting | Yes | Yes |
Core Modules | | |
Regular Sleep-Wake Times | Yes | Yes |
Wind-Down Routine | Yes | Yes |
Wake-Up Routine | Yes | Yes |
Improving Daytime Functioning | Yes | Yes |
Unhelpful Beliefs about Sleep | Yes | No |
Maintaining Your Gains | Yes | Yes |
Optional Modules | | |
Reducing Sleep-Related Worry | Yes | Yes |
Improving Sleep Efficiency | Yes | No |
Reducing Time in Bed | Yes | No |
Delayed or Advanced Phase | Yes | No |
CPAP Machine and Exposure | Yes | No |
Negotiating Complicated Environments | Yes | No |
Reducing Nightmares | Yes | No |
Note. The core modules of Regular Sleep-Wake Times, Wind-Down Routine, and Wake-Up Routine are delivered as one core module in Standard TranS-C and three core modules in Adapted TranS-C. “Yes” indicates that the module is included in the given version of TranS-C. “No” indicates that the module is not included in the given version of TranS-C. |
Standard TranS-C
Standard TranS-C is delivered via eight, 50-minute weekly sessions and comprised of four cross-cutting modules featured in every session, four core modules delivered to most patients, and seven optional modules that are used based on clinical presentation, treatment goals, and provider case conceptualization [36]. Initial training for providers in the Standard TranS-C condition consists of a 1-day workshop (i.e., six to eight hours) or two, 3-hour training blocks, based on CMHC preferences.
Adapted TranS-C
Adapted TranS-C is delivered via four, 20-minute weekly sessions and comprised of the same four cross-cutting and core modules as in Standard TranS-C. The core modules are divided into five (rather than four) modules. Additionally, there is one optional module that can be integrated with the core modules, based on clinical presentation, treatment goals, and provider case conceptualization. Training for the Adapted TranS-C condition consists of four, 1-hour workshops or two, 2-hour workshops, based on CMHC preferences.
The process of adapting TranS-C was grounded in theory, data, and stakeholder input. As the overarching guide for the adaptation process, the Replicating Effective Programs framework [13] was used. To summarize, the following were considered for REP: the need and evidence for TranS-C [29]; data from interviews with stakeholders [32, 37]; a pilot study of Adapted TranS-C (unpublished data); and TranS-C’s theoretical underpinnings and mechanisms of action [23, 36]. Additionally, following adaptation and treatment development frameworks, Adapted TranS-C was designed for a broad range of patient and implementation characteristics (e.g., symptom severity; CMHC resources) [38]. Additional details about the adaptation process are described elsewhere [33].
Facilitation: Implementation, Train-the-Trainer, and Sustainment Phases
During the Implementation and Train-the-Trainer Phases of the three-phase trial, facilitation was used as the core implementation strategy, based on the Enhanced Replicating Effective Programs framework [39] and promising evidence [40, 41]. Specifically, each CMHC received direct support from a lead facilitator (ERA)—who also served as the expert TranS-C trainer—as well as a team of trained facilitators, all of whom were employed by the research team and supervised by the Principal Investigator (AGH).
In the Implementation Phase, the external facilitators supported implementation of TranS-C in participating CMHCs through a range of activities, including leading TranS-C trainings, distributing TranS-C manuals and other educational materials, holding weekly TranS-C supervision and as-needed consultation, problem solving administrative barriers such as negotiating productivity requirements and ensuring that TranS-C activities counted toward Continuing Education Unit credits (CEUs), offering sleep treatment certification, and collaborating with leadership, key providers, and site champions. Additional details about the Implementation Phase are reported elsewhere [33].
The facilitation team transitioned CMHC sites to the Train-the-Trainer Phase on a rolling basis. The first site was transitioned to the Train-the-Trainer Phase in December 2020, and all sites were transitioned by December 2022 [33, 34]. During the Train-the-Trainer Phase, the facilitators engaged in the following: recruiting, training, and providing consultation for local CMHC trainers; recruiting and enrolling providers and patients; holding as-needed consultation for TranS-C providers; offering certification in sleep treatment and sleep training; processing CEUs; and organizing regular meetings with CMHC leadership to problem-solve barriers. Additionally, as CMHC providers and trainers gained mastery and independence, the facilitation team gradually transferred the following responsibilities to them: TranS-C trainings, clinical supervision, presentations on advanced topics to other providers, and cross-county consultation among trainers (termed the “Sleep Expert Network”). Additional details about the Train-the-Trainer Phase are reported elsewhere [34].
Sites were transitioned from the Train-the-Trainer Phase to the Sustainment Phase between January 1, 2023 to June 1, 2023. When transitioning CMHC sites to the Sustainment Phase, facilitators considered several factors. These factors included patient and provider recruitment, CMHCs’ established procedures to sustain TranS-C (e.g., scheduled TranS-C trainings on the calendar), CMHC provider and trainer mastery, and support from CMHC leadership. For each site, facilitators drafted individually-tailored sustainment plans, which consisted of detailed checklists to help leadership, providers, and trainers establish systems to support continued delivery and training in TranS-C in the following three domains: (1) providing clients with TranS-C, (2) training and supporting TranS-C providers, and (3) identifying, training, and supporting TranS-C trainers. When a site completed most or all items on their sustainment plan, the facilitators held a sustainment meeting with CMHC leadership, providers, and/or trainers to answer any final questions. After this meeting, the site officially graduated to the “Sustainment Period” during which the site received minimal facilitation support for three months. Note that three months were selected for the Sustainment Period following research precedent that clinics may be at risk of sustainment failure as early as three months after implementation efforts have ended [42]. Following the Sustainment Period, providers are recruited for Sustainment Phase data collection (see Fig. 2), which began in May 2023. Depending on recruitment progress, sustainment data collection may continue for up to approximately one year (i.e., through March 2024).
Here we note that the initial plan was to withdraw all support during the Sustainment Phase. However, upon deliberation and consultation with CMHC partners and experts in clinical service implementation and delivery, we decided that facilitators could continue (a) treatment-related assistance that is typically sought from outside experts or entities in clinical settings, and (b) minimal background support. The treatment-related assistance consists of: provider-initiated informal consultation with facilitators (e.g., akin to “curbside consultation” with external experts); organizing CEUs and sleep coaching certification (e.g., akin to an outside institution offering CEUs or EBPT certification); and sending workbooks and manuals to counties as needed but no more than once per month (e.g., akin to an outside organization offering limited free treatment-related resources). The minimal background support consists of: gathering recordings of TranS-C trainings led by CMHC trainers and sitting in on presentations on advanced sleep-related topics and Sleep Expert Network meetings. This ongoing minimal background support is provided for two reasons: (1) to enable continued data collection (e.g., to compare training techniques of expert facilitators relative to CMHC trainers), and (2) to preserve community partnerships (e.g., sitting in on CMHC providers’ presentations to help them feel supported and encouraged).
Measures
The measures described below are organized by the three domains of Shediac-Rizkallah and Bone’s sustainment framework (i.e., activities, benefits, and capacity [12]) followed by the proposed mechanism (i.e., provider perceptions of fit). Activities are operationalized as providers’ delivery, adaptations, and routinization of TranS-C in clinical practice [12]. Benefits are operationalized as providers’ assessment of TranS-C’s health benefits for patients [12]. Capacity is operationalized as providers’ knowledge, skills, and resources to deliver TranS-C [12]. Provider perceptions of fit are operationalized as TranS-C’s perceived acceptability, appropriateness, and feasibility [43–45].
Only measures that will be analyzed for the primary aims of the Sustainment Phase are reported below. Note that for all measures below, providers are asked to consider only the time since their CMHC graduated to the Sustainment Phase. For all relevant questions, providers are offered a “not applicable” option if they have not delivered TranS-C during the Sustainment Phase. For some measures, the language was modified slightly from the original measure to increase accessibility and relevance for providers (e.g., changing “intervention” to “sleep treatment”).
Provider Characteristics
Providers are asked to report their degree, theoretical orientation, age, sex assigned at birth, gender, ethnicity, and race.
Activities: Delivery, Adaptations, and Routinization
Primary Outcomes. The Provider REport of Sustainment Scale assesses providers’ continued delivery of TranS-C [46]. It was designed as a brief, pragmatic measure for direct service providers to report their continued use of a given evidence-based practice. The measure consists of three items that are rated on a scale from 0 (not at all) to 4 (to a very great extent), with higher scores indicating more sustainment, and has demonstrated acceptable internal consistency reliability (Cronbach’s alpha = 0.95; McDonald’s omega = 0.95) and construct validity within a similar sample of providers [46].
The Adaptations to Evidence-Based Practices Scale [47] assesses provider adaptations to treatment. Providers are asked to rate six items using a 4-point Likert scale from 1 (not at all) to 4 (very great extent)2. Each item assesses the extent to which providers have made a specific type of adaptation during the Sustainment Phase (e.g., modifying presentation, shortening or condensing pacing, removing or skipping components), and higher scores indicate greater use of adaptations. This measure has demonstrated acceptable reliability and construct validity within a similar sample of providers [47].
Penetration is assessed following a widely-used definition and formula [48]. Specifically, providers are asked to report (a) how many of their patients have had sleep problems during the Sustainment Phase and (b) the number of those patients with whom the provider has used TranS-C during the Sustainment Phase. Then, “b” is divided by “a,” with higher scores indicating more penetration. In a review of implementation science measures, this formula was found to have excellent usability [49].
Secondary Outcomes. As a proxy for change in TranS-C delivery, providers are asked whether they are using TranS-C “more,” “about the same,” or “less” relative to before the Sustainment Phase. As another secondary measure of delivery, providers complete the TranS-C Provider Checklist (modified for providers in the Adapted TranS-C condition to include only the relevant modules) [50]. On this measure, providers identify the cross-cutting, core, and optional modules they have delivered during the Sustainment Phase. The TranS-C Provider Checklist has demonstrated acceptable internal consistency (Cronbach’s alpha = 0.74, mean interitem correlation ρ = 0.16 ) and convergent validity among university-hired therapists delivering TranS-C at a CMHC [50]. 3
As a secondary measure of adaptations, providers complete a checklist from the coding manual for the Framework for Reporting Adaptations and Modifications – Expanded [51]. On this checklist, they indicate whether they have made any adaptations to TranS-C during the Sustainment Phase based on patient characteristics, such as race or ethnicity, gender identity, first/spoken language, literacy and education level, comorbidity/multimorbidity, and motivation and readiness. As additional secondary measures of adaptations, providers are asked (a) to rate the extent to which they follow the TranS-C provider manual and patient workbook on a scale from 0% (not at all) to 100% (always/completely); (b) to estimate the percentage of TranS-C strategies they use on average when delivering TranS-C to their patients on a scale from 0% (no strategies) to 100% (all the strategies); (c) to indicate whether they have created their own sleep treatment materials, such as sleep diary, worksheet, or video (response options: no/not relevant/yes with option to describe); (d) to indicate whether they deliver TranS-C as a standalone intervention or whether they integrate it with other interventions/topics (e.g., for mental health, medication, housing, vocational training) (response options: I always deliver the sleep treatment as a stand-alone intervention; I sometimes integrate the sleep treatment with other interventions or topics; I always integrate the sleep treatment with other interventions or topics; not applicable); and (e) to report the number of sessions in which they use TranS-C concepts for each patient, on average.
Benefits: Perceived Health Benefits for Patients
Primary Outcome. The Outcomes & Effectiveness Scale is a 5-item scale from the Clinical Sustainability Assessment Tool [52], which assesses providers’ perceptions of TranS-C’s health benefits. Items are rated on a scale from 0 (to little or no extent) to 7 (to a very great extent)4 and higher scores indicate more perceived benefits. This measure has demonstrated acceptable construct validity in similar contexts, and the subscale has demonstrated satisfactory internal consistency reliability (Cronbach’s alpha = 0.93) [52].
Capacity: Knowledge, Skills, and Resources
Primary Outcomes. The Skills Subscale from the Determinants of Implementation Behavior Questionnaire [53] assesses providers’ perceptions of their skills to deliver TranS-C. Three items are rated on a scale from 1 (strongly disagree) to 7 (strongly agree), where higher scores indicate more skills. This scale has demonstrated good discriminant validity and internal consistency reliability (Cronbach’s alpha = 0.86) [53, 54].
The Organizational Resources Subscale from the Implementation Potential Scales [55] is used to assess providers’ perceptions of whether they have the resources, support, and time needed to deliver TranS-C. Three items are rated on a scale from 1 (strongly disagree) to 6 (strongly agree), where higher ratings indicate more perceived resources, support, and time. This scale has demonstrated good construct validity and internal consistency reliability (Cronbach’s alpha = 0.85) in a sample of school psychologists [55].
Secondary Outcomes. The Administrator Support Subscale from the Implementation Potential Scales [55] is used to assess the extent to which providers perceive they have support from leadership and supervisors to deliver TranS-C. Three items are rated on a scale from 1 (strongly disagree) to 6 (strongly agree), where higher scores indicate more support. This scale has demonstrated good construct validity and internal consistency reliability (Cronbach’s alpha = 0.86) in a sample of school psychologists [55].
Two knowledge items, modeled on Kauth et al. [40], are rated on a scale from 0 (not at all) to 7 (extremely), where higher scores indicate more knowledge. Specifically, providers rate the extent to which they (1) understand the theory and concepts behind TranS-C and (2) have the knowledge to conduct TranS-C.
Proposed Mechanism: Provider Perceptions of Fit
Providers rate the acceptability, appropriateness, and feasibility of TranS-C via the following measures: Acceptability of Intervention Measure, Intervention Appropriateness Measure, and Feasibility of Intervention Measure [56]. Each of these measures consists of four items that are rated on a scale from 1 (completely disagree) to 5 (completely agree), where higher scores indicate greater acceptability, appropriateness, and feasibility, respectively. These measures have demonstrated satisfactory validity and reliability in a convenience sample of mental health counselors [56].
Semi-Structured Interview
Providers are invited to complete a semi-structured interview that consists of 11 questions, each of which focuses on one of the following domains per Shediac-Rizkallah and Bone’s sustainment framework [12]: continued TranS-C activities, perceptions of TranS-C’s continued benefits to patients, and continued capacity to deliver TranS-C. Pre-specified as well as impromptu probes are used to assess possible mechanisms of outcomes. The first author (LDS) drafted the interview, then questions were refined with input from the facilitators, collaborators, and members of the research team. Following recommendations by implementation science experts [57], qualitative methods are included to gather more in-depth information about sustainment outcomes and mechanisms from the perspective of providers.
Procedure
The sustainment surveys and interview are delivered one time at least three months after graduation to the Sustainment Phase (i.e., after the Sustainment Period). Before participating in this study, all providers give informed consent via secure, online forms (Docusign or Qualtrics) and are informed that they can withdraw from the study at any time. As noted above, providers are compensated according to their CMHC’s policy. Throughout all phases of the trial, providers are blind to treatment condition (i.e., Standard or Adapted TranS-C).
The surveys are compiled into a single assessment battery and administered on a version of Qualtrics that is compliant with the Health Insurance Portability and Accountability Act (HIPAA). Semi-structured interviews are delivered by the first author, members of the research team, and the lead facilitator via phone or HIPAA-compliant Zoom. Interviewers are not blind to treatment condition to enable asking appropriate probes during the interview. However, interviewers are thoroughly trained to deliver the interviews with integrity and minimal bias. The first author listens to interview recordings and provides feedback, and group/individual supervision is provided as needed.
Allocation
During the Implementation Phase, CMHCs were randomized to Standard or Adapted TranS-C through a computerized randomization sequence by a biostatistician with no stratification at the CMHC or provider level. Throughout all phases of the trial, sites retain their original randomization assignment to Standard or Adapted TranS-C. Only the facilitators and research team (i.e., not CMHCs or providers) are privy to which CMHCs and providers are allocated to which TranS-C treatment condition (Standard TranS-C or Adapted TranS-C).
Sample Size
The number of providers for the quantitative analyses in this study (N = 154; 140 plus 10% for dropout) was selected based on recruitment from the Implementation Phase. Sample size determination was not needed for Aim 1, which consists of descriptive statistics (see Analysis Plan below). For Aim 2, using this sample size in a cluster-randomized trial design, a minimum detectable effect size was calculated using Stata [58, 59]. Prior studies have reported moderate to large correlation coefficients between sustainment outcomes (rs = 0.34–0.64) [46, 60]. Based on the site intra-class correlation (ICC) estimated from similar prior studies [39, 61], the ICC was assumed to be 0.01. The coefficient of variation of cluster size was calculated as 0.31, based on the ratio of standard deviation of cluster size to mean cluster size [62]. A two-sided alpha of 0.05 was used. Together, the minimum detectable effect size using a cluster-randomized design with a sample of 140 across 10 clusters was a small to medium effect size of d = 0.40. Given that a prior study with a similar aim and outcomes produced a large effect size [63], we expect that it will be feasible to detect a small to medium effect size. For Aim 3, a Monte Carlo power analysis through Schoemann et al.’s [64] application was conducted for parallel mediators with 1,000 and 5,000 replications, 20,000 Monte Carlo draws per replication, and 95% confidence intervals per recommendations. Drawing from prior research, large correlations (r = 0.50) were assumed between: the predictor (Standard vs. Adapted) and mediators (acceptability, appropriateness, feasibility) [63]; the mediators (r = 0.50) [56, 65]; and the mediators and sustainment outcomes (r = 0.50) [17–19]. Moderate correlations (r = 0.30) were assumed between the predictors and outcomes [19]. Given these estimates, the power detected for the indirect effects with a sample size of N = 140 was high (0.92–0.98).
For qualitative analyses, the target sample size (N = 40; n = 20 providers from Standard TranS-C, n = 20 providers from Adapted TranS-C) was guided by findings that saturation can be reached with an upper bound of 17 interviews [66].
Data Management and Dissemination
All patient-identifiable data are saved on a secure password-protected and HIPAA-compliant website. After data have been collected, provider-identifiable data are removed and providers are assigned identification numbers. Participant-identifiable data are not shared with outside entities during or after the trial. The first author is responsible for downloading, collating, and analyzing the data.
A Data Safety Monitoring Board has been formed to help prevent and manage adverse events. The board includes members with expertise in SMI, psychosocial treatments, and randomized controlled trials. Members are independent from the first author (LDS), Principal Investigator (AGH), and competing interests. A report was made to the board bi-annually for the first year of the research conducted during the Implementation Phase (Phase 1). Since then, the schedule has shifted to annual reports. However, if safety issues arise, the schedule will be changed to monthly reports. Yearly reports are submitted to the Committee for the Protection of Human Subjects at University of California, Berkeley and the National Institute of Mental Health (NIMH). Triyearly reports on recruitment are also submitted to NIMH.
During the Sustainment Phase, interim analyses are not conducted. Results from the trial, as well as analysis code, will be shared via peer-reviewed publications, professional conference presentations, and meetings and newsletters to CMHCs, as relevant. Other than the authors and compliance with data-sharing agreements stipulated by the National Institutes of Health, no other entities have contractual agreements to access the final dataset. Deidentified data are submitted to the NIMH Data Archive twice per year, per NIMH requirements.
Roles and Responsibilities
This study is led by the first author (LDS), who supervises the Sustainment Phase research team and is responsible for data management, under the general supervision of the larger trial’s Principal Investigator (AGH), who also manages the facilitation team. The Principal Investigator and other collaborators offer expert guidance. The research team is responsible for the informed consent process, recruiting providers, and collecting data. The first author is responsible for downloading, collating, and analyzing the data. The facilitators offer the minimal support to CMHCs during the Sustainment Phase, as described above. Members of all these teams will collaborate on writing up and disseminating the data. Communication occurs through as-needed meetings and regular email communication. There is no coordinating center, trial steering committee, Data Monitoring Committee, or Stakeholder and Public Involvement Group. The trial sponsor is University of California, Berkeley.5 Other than ethical approval for the study, the sponsor has no role or authority in study design; collection, management, analysis, or interpretation of the data; writing of the report; or the decision to submit the report for publication. With respect to audits, organizations not directly involved in the trial (e.g., NIMH, Committee for the Protection of Human Subjects, Data Safety Monitoring Board) have the right to audit and, if such a situation arises, will determine the frequency and procedures for auditing. The first author regularly audits recruitment and quality of data collected.
Planned Analyses
Quantitative Analyses
Preliminary Analyses and Missing Data
Analyses will use all available data [67] and maximum likelihood estimation. Patterns of missingness will be investigated by conducting Little’s MCAR test and testing the extent to which missingness is related to observed variables [68]. Baseline between-group differences in demographic variables will be examined and considered as possible covariates (e.g., depending on relationships to predictors and outcomes) [69]. Distributions will be evaluated to detect outliers, and we will ensure that the assumptions of planned analyses are met. ICCs will be reported.
Aim 1. Report TranS-C Sustainment Outcomes
Descriptive statistics of all primary and secondary sustainment outcomes (e.g., mean, standard deviation, range, frequency, percentage), as well as provider characteristics, will be reported.
Aim 2. Treatment Condition on Sustainment Outcomes
Hierarchical linear modeling with maximum likelihood estimation will be used to test the effect of TranS-C condition (Standard vs. Adapted TranS-C) on primary sustainment outcomes, while accounting for providers (Level 1) nested in CMHCs (Level 2) [68]. The predictor will be represented by a dummy-coded variable for condition (1 = Adapted, with Standard as the reference group), and all outcomes will be modeled as continuous. The parameters of interest will be the effect of condition (Standard vs. Adapted TranS-C) on the primary sustainment outcomes.
Aim 3. Fit as a Mediator of Treatment Condition and Sustainment Outcomes
Using structural equation modeling, multivariate parallel mediation models will test whether provider perceptions of acceptability, appropriateness, and feasibility mediate the relations between condition (dummy-coded as Adapted = 1, with Standard as the reference group) and primary sustainment outcomes (all continuous). Models will adjust for cluster (i.e., CMHCs). Three mediation models will be tested: one model will be evaluated for each category of sustainment outcomes (i.e., activities, benefits, capacity), and each model will simultaneously evaluate the three measures of fit (i.e., provider perceptions of acceptability, appropriateness, and feasibility).
Mixed Methods Analyses
Interviews will be coded and analyzed using thematic analysis [70] with a combination of deductive and inductive approaches [71], after which qualitative findings will be triangulated with survey data [72]. Interviews will be recorded and transcribed verbatim. The first author (LDS) will lead a coding team with expert input from other authors. Each coder will be required to establish 80% or higher inter-coder agreement with the first author across five interviews. The coding team (other than the first author) will be blind to provider condition (Standard vs. Adapted) and study hypotheses.
Deductive and inductive codebooks will be developed to guide data coding. The deductive codebook will consist of sustainment outcomes according to Shediac-Rizkallah and Bone’s sustainment framework (i.e., continued activities, benefits, and capacity) [12]. The lead coder will develop the inductive codebook by reading through all transcripts to become familiar with the data, then rereading the transcripts to identify inductive codes that emerge related to sustainment outcomes as well as possible predictors and mechanisms of sustainment outcomes (e.g., provider perceptions of fit) [70].
After the data have been deductively and inductively coded by the coding team, the first author will review the coded data, during which themes present in the interviews will be identified and refined [70, 71]. These themes will be used to analyze the extent to which sustainment outcomes were met, with respect to continued activities, benefits, and capacity (i.e., to supplement Specific Aim 1). Next, themes will be compared across the Standard and Adapted TranS-C conditions to help determine whether manipulating treatment fit to context impacts sustainment outcomes (i.e., to supplement Specific Aim 2). Additionally, themes will be analyzed to assess possible predictors and mechanisms of outcomes (i.e., to supplement Specific Aim 3). Triangulation will be conducted using a concurrent approach, in which interviews and surveys will be analyzed at the same time and given equal weight during interpretation [72]. Triangulation will be used to analyze the extent to which data converge, as well as to offer a deeper analysis of sustainment outcomes, predictors, and mechanisms [72, 73].
- Note that the vast majority of providers were employed by CMHCs. However, some providers left their CMHC position during the study. Also, in very few isolated instances, providers outside of CMHCs learned about the study (e.g., by word of mouth) and requested to participate. When the latter providers otherwise met the criteria, they were permitted to participate and were matched with a CMHC patient by the facilitation team. For consistency across phases of the trial and to maximize sample size, these providers (i.e., who left their CMHC after receiving TranS-C training or requested to participate from outside of CMHCs) will be included in Sustainment Phase recruitment efforts.
- The original measure used a 5-point scale from 0 to 4. However, this measure was delivered to providers using a 4-point scale at earlier timepoints in the present trial; thus, a 4-point scale was used for consistency.
- In the original measure [50], the nightmares module was not included due to administrative error. It was added to the version administered to providers in Standard TranS-C (Adapted TranS-C does not include the nightmares module).
- Due to administrative error, this scale deviates from the original scale of 1 to 7.
- Contact information for Sponsor: +1(510) 642–6000; University Avenue and Oxford Street, Berkeley, CA 94702.