Service providers endorse integrated services model for youth with mental health and substance use challenges: findings from a discrete choice experiment

Background Given high rates of mental health and substance challenges among youth and substantial system access barriers, system innovation is required. Integrated youth services (IYS) models aim to transform youth mental health and substance use services by creating integrative, collaborative models of care in youth-friendly settings. This study examines service provider perspectives on the key service components to include in IYS models. Method A discrete choice experiment modeled service provider preferences for the service components of IYSs. The sample includes 388 service provider/agency leader participants (age 18+) from youth-serving organizations in Ontario. Importance scores and utility values were calculated for 12 attributes represented by four levels each. Latent class analysis identified subgroups of participants with different preferences. Results The majority of participants were direct service providers working in larger organizations in the mental health and/or substance use sectors in large urban centers. Participants strongly endorsed service models that provide rapid access to the widest variety of culturally sensitive service options, with supplementary e-health services, in youth-focused community settings with evening and weekend hours. They prefer caregiver involvement in youth services and treatment decisions and support youth and family engagement. Latent class analyses reveal three segments of service providers: a Youth-Focused Service Accessibility segment representing 62.1% (241/388) of participants, a Service Options segment representing 27.6% (107/388) of participants, and a Caregiver Integration segment representing 10.3% (40/388) of participants. Within these segments, the degree of prioritization of the various service components differ; however, the overall endorsement of the service components remains largely consistent across classes for most attributes. The segments did not differ based on demographic or agency characteristics. Conclusions The core characteristics of IYS settings for youth with mental health and substance use challenges, i.e., rapid access to a wide range of youth-oriented services, are strong priorities of service providers and youth-serving agency leaders. These findings confirm that youth-oriented service providers endorse the importance and relevance of IYS models as a whole; strong service provider buy-in to the model is expected to facilitate development, implementation and scaling of IYS models. Hearing stakeholder perspectives, including those of service providers, youth, and caregivers, is essential to developing, effectively implementing, and scaling effective youth services. Supplementary Information The online version contains supplementary material available at 10.1186/s12913-021-07038-3.


Conclusions:
The core characteristics of IYS settings for youth with mental health and substance use challenges, i.e., rapid access to a wide range of youth-oriented services, are strong priorities of service providers and youthserving agency leaders. These findings confirm that youth-oriented service providers endorse the importance and relevance of IYS models as a whole; strong service provider buy-in to the model is expected to facilitate development, implementation and scaling of IYS models. Hearing stakeholder perspectives, including those of service providers, youth, and caregivers, is essential to developing, effectively implementing, and scaling effective youth services.
Keywords: Youth, Mental health, Substance use, Integrated services, Service provider Background Adolescence and young adulthood is a critical period for the emergence of mental health and substance use challenges [1,2]. Experiencing mental health or substance use challenges during the adolescent or emerging adult period is associated with multiple challenges in terms of immediate functioning and wellbeing, as well as longterm development and the transition to a productive adulthood [3,4]. Recent epidemiological research shows that some 7.8% of Canadian youth have been diagnosed with a mood disorder, and 12.9% with an anxiety disorder [5]. Rates of diagnosed disorders among youth, self-reported dissatisfaction with their mental health, and service seeking, have been increasing over the past years [5]. Prescription opioid drug use for non-medical reasons is reported by 11% of students in grades 7 to 12 over the past year, while some 14% of youth in grades 9 to 12 report hazardous drinking and over 3% report signs of cannabis dependence [1].
Given the high rates of mental health and substance use challenges during this critical developmental period, access to high quality services is paramount. Unfortunately, the youth mental health service system is suboptimal. Long wait lists, complicated pathways to care, a lack of specialized services, stigma, and unresponsiveness constitute significant service access barriers [6][7][8][9][10][11]. Indeed, the majority of youth with mental health challenges do not access them [12]. Among those who do access services, dropout rates are high [13,14], missing the opportunity to provide the support young people need to get back on track and show improvements that can have lifelong impacts.
To better address youth mental health needs, transformative system change is required. Reflecting this, a systems transformative movement is under way around the world, involving the creation of Integrated Youth Services (IYS) models of service delivery [15,16]. The recently emerging IYS models aim explicitly to provide rapid access to effective services in a one-stop-shop model of care, where youth can access services for a wide range of health and psychosocial support needs in responsive, youth-friendly settings [17][18][19][20]. IYSs bring together service providers across a broad range of disciplines, such as psychotherapy, psychiatry, peer support, employment support, primary care, and other disciplines, who work in an integrated manner. Examples of IYS models emerging in Canada include the Foundry model in British Columbia, the pan-Canadian Access Open Minds model, and the Youth Wellness Hubs Ontario model [19][20][21].
Service integration in the Canadian youth mental health, substance use, and social services landscape has been inadequate in recent years, which leaves youth underserved in areas of care that are outside of the area of expertise of the organization they have accessed [22]. IYS models aim to remedy this gap in the system to provide the best available care for youth across a wide variety of areas of need. While recent work has identified the core components of IYS settings as a whole and around the world [15], it is not clear which components are most crucial to IYS service design relative to other components; this is an essential step in identifying which components should be prioritized when faced with limited resources and competing priorities.
The implementation science literature demonstrates the importance of taking into account stakeholder perspectives when implementing complex health interventions [23]. Service provider perspectives are particularly important, as interventions with strong service provider buy-in are more likely to be implemented effectively [24]. In the IYS model, it is particularly important to take service provider perspectives into account given the integration component, which brings together a wide variety of service providers from different backgrounds who may not be accustomed to working together in an integrated manner. Other stakeholder perspectives are also critical to building appropriate services. Notably, the perspectives of youth and the caregivers who support them must be taken into account to build services that are appealing to them, and are therefore more likely to be accessed by youth in need [25,26].
This research project takes a rigorous approach to understanding the perspectives of multiple stakeholders in the development of IYS models of service delivery. Moving beyond the efficacy and effectiveness literature addressing specific treatments to be embedded into IYS settings, this study examines other diverse aspects of the setting that are not subject to efficacy trials, e.g., fundamental aspects of the setting such as the diversity of services, rapid access, hours of availability, and engagement. A scoping review [27] mapped various components of IYS settings around the world, and found that many components were similar across models, but some differed, such as the types of service providers; however, a lack of reporting on a number of aspects of care prevents drawing conclusions about the consistency of their implementation. To understand which components are the most important to implement in IYS settings, we used a discrete choice experiment (DCE) methodology to identify the relative importance of various setting characteristics as compared to others. We sought the perspective of service providers, youth, and caregivers on the key components of IYS services]. This paper presents the findings for the service provider sample; youth and caregiver perspectives are presented in companion manuscripts [28,29].
Research questions. This study aimed to answer the following research questions: 1) What service characteristics do service providers consider the most important in building integrated youth services? 2) Do different segments of service providers exist, defined by different preferences? If so, what service characteristics and participant attributes define these segments?

Method
The DCE is a quantitative approach that elicits consumer preferences regarding products or services with complex sets of hypothetical characteristics. DCE puts different characteristics ("attributes") of a service or product head to head in hypothetical scenarios and asks the respondent to choose between them. The DCE makes it possible to determine the relative importance of different priorities over others. By combining multiple attributes represented by various levels of that attribute, it becomes possible to determine participants' relative preference for one alternative over another in a complex set of service options. This methodology also identifies consumer subgroups with different sets of preferences. This DCE was designed following the International Society for Pharmacoeconomics and Outcomes Research taskforce's report on Good Research Practices for Conjoint Analysis [30].
First, a scoping review was conducted to identify attributes and levels of possible services and service delivery options for an IYS [27]. Using this, draft attributes were created in consultation with the project team through iterative discussions, as well as opportunities to provide comments and rank their perspectives of the importance of items via project development surveys; these were programmed in REDCap [31,32] software and distributed to the project team, made up of research team members (including community partners and youth) with substantial experience working in the IYS sphere. Feedback was used to progressively refine the attributes and levels. The resulting DCE survey was piloted with four youth, four caregivers, and two service providers in two locations, one large Canadian city and one rural area in Ontario, Canada.
Feedback from each of these phases led to the finalization of the DCE, which contained 12 attributes of 4 levels each. The complete list of attributes and levels used in this study is presented in Table 1. The final DCE survey was programmed into Sawtooth Software's SSI Web (version 9.8) and administered using that platform. The choice tasks utilized a partial profile design in which participants were asked to choose between three service options. Participants were asked to think about services for youth (aged [14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29] with mental health and/or substance use challenges and to select the best service option from among the three presented. The presented service options contained levels of three attributes; for a sample item, see Fig. 1. Surveys were balanced by an algorithm that optimized orthogonality and attribute/level balance. Each participant randomly received a different version of the survey.

Participants & procedure
Service provider participants were recruited from a comprehensive internal database of organizations offering services to Ontario youth with mental health and substance use challenges. Diverse perspectives were sought from service providers offering services in various sectors. Organizations were contacted by email with a link to the online survey and encouraged to disseminate that link to the service providers and organization leaders in their network. After completing informed consent, eligibility questions were presented; participants were deemed eligible if they either provided direct service to Ontario youth, or were in a leadership/management or administrative role for an agency that provided service to Ontario youth. An explanation of the DCE process was then provided, with a practice question prior to the DCE. Participants who completed the survey had the option of following a separate link to enter a draw to win one of a series of gift cards as a form of compensation; the draw entry was unlinked to the DCE survey and DCE responses were anonymous. The survey was conducted in English. Response time was a median of 15 min. The final sample size for the study was 388 participants. The study received approval from the Centre for Addiction and Mental Health Research Ethics Board.

Other measures
In addition to the DCE, participants were asked demographic questions (e.g, age, gender, education, years of experience). They were also asked about their professional background, the primary and secondary sectors in which they offered care to youth, the age of the youth they served, and other descriptors of their roles.

Data analysis
Descriptive statistics were calculated to describe the sample. The DCE was analyzed by estimating utilities for each participant using hierarchical Bayesian methods performed using Sawtooth Lighthouse Studio (9.8.1, [33]). Standardized, zero-centered utilities were used [34], with higher utility values reflecting the relative value of that level in relation to the other attribute levels; positive utility values represent a positive relative preference in relation to other attribute levels, while negative utility values reflect a negative relative preference. Each attribute's relative importance was assessed by calculating the proportional utility of each of these attributes relative to the total utility provided by all attributes. Attributes that are selected more often by participants feature higher importance values.
Latent class analysis was employed to identify segments of participants with similar service preferences. Each participant was assigned a probability of belonging to a particular segment. Latent class analyses were performed using Latent Class module in Sawtooth Software. Each latent class solution was replicated five times at

Core Health Services
Only mental health counseling.
Mental health and substance misuse counseling.
Mental health and substance misuse counseling, and medication management.
Mental health and substance misuse counseling, medication management, and physical/sexual health.

Other Services
Education and employment services.
Housing, shelter and income support services.
Legal support services. Choice of education, employment, housing, income support, and legal support services.

Caregiver Involvement
No caregiver involvement. Caregivers receive own counseling.
Caregivers involved in family counseling with youth, with youth consent.
Caregivers involved in decisions regarding youth counseling, with youth consent.

Peer Support
Recreational activities led by trained peer support worker.
Can talk to a trained peer support worker, upon request.
Mental health groups run solely by a trained peer support worker.
Youth can be matched to an ongoing trained peer support worker to learn life skills and help them with services they need.

Cultural Sensitivity
Cultural background is not considered when picking a service or service provider.
Can ask for a service provider with a certain cultural background, when available.
Services are culturally sensitive and trauma-informed.
Culturally based services are available for cultures common in the local area.

E-Health Services
No e-health or electronic services.
Can schedule or reschedule appointments via email, text or online.
E-health services are offered 24/7 alongside in-person services during office hours.
All services are delivered only through a website, e-mail, text, or phone app.
7. Age Range Services for ages 12-24, in a youth-only setting.
Services for ages 12-29, in a youth-only setting.
Services for ages 12-24, in a setting that also has services for children 0-12.
Services for ages 12-29, in a setting that also has services for adults 29 + .

Time of Appointments
Monday to Friday, 9 AM-5 PM.
Monday to Friday, 9 AM-9 PM, and Saturday, 9 AM-5 PM. See a counselor for the first time after more than 1 month.

Location
Building or office in the community that specializes in mental health services.
Youth cafe and recreation centre.
Hospital or doctor's office. School setting.

Engagement
Youth and caregivers give feedback, e.g., anonymous surveys.
Youth and caregivers are on staff at the organization.
Youth and caregivers are on an advisory group that gives feedback on services and evaluation.
Youth and caregivers play a leadership role in making decisions for the organization.

Information Sharing
No sharing of personal information with caregivers.
All personal information is available to caregivers, with youth consent.
Service provider decides what information to share with caregivers, with youth consent.
Youth and service provider work together to decide what personal information to share with caregivers and how that can be helpful.
different starting seeds. Convergence was assumed when log-likelihood decreased by 0.01 or less. Based on the selected model, attribute rankings are presented descriptively. Chi-square tests were used to compare demographic characteristics across the segments. All other analyses were performed using SPSS 25 [35].

Results
Participant characteristics are presented in Table 2. Participating service providers were highly likely to be female and be direct service clinicians (as opposed to in leadership or administrative positions). They were mostly from large urban centres, tended to have been their current position for five years or less, and were likely to have worked for large agencies with 30 or more employees. The majority provided services in the mental health and/or substance use sectors. The overall preferences for participants as a whole are presented in Table 3. Notably, participants positively endorsed offering the widest possible array of services, including within the 'Core Health Services' attribute, as well as supplementary services outside of the mental health and substance use spheres ('Other Services' attribute) and including trained peer support workers ('Peer Support' attribute). A rapid access model was also preferred ('Wait Times' attribute). They preferred offering culturally sensitive, trauma informed services ('Cultural Sensitivity') and providing supplementary e-health services and electronic scheduling ('E-health Services' attribute). Offering evening and weekend hours was positively endorsed, with some endorsement of 24/7 services ('Time of Appointments' attribute). Situating services in a community service setting or a youth café and recreation centre was preferred ('Location' attribute). Participants endorsed family counseling and involving caregivers in decisions ('Caregiver Involvement' attribute) as well as working with the youth to determine what information to share with caregivers ('Information Sharing with Caregivers'). In addition, participants positively endorsed engaging youth and caregivers in an advisory group ('Engagement' attribute). For the 'Age Range' served attribute, the preferences were youthfocused services, but with low relative importance. The highest relative importances were for information sharing with caregivers, the variety of supplementary services, and supplemental e-health services.
Latent class analysis was conducted on the DCE items. Fit indices are presented in Table 4 for segment sizes of two to five. The three-segment model was selected based on a combination of fit, segment size, and interpretability. The attribute rankings for each of the three segments are presented in Table 5.  The first segment, made up of 241 (62.1%) participants, endorsed services that meet the access needs of youth, leading to this segment being labelled Youth-Focused Service Accessibility. This segment strongly endorsed the Cultural Sensitivity, Wait Times, E-Health Services, and Core Health Services attributes. The attributes endorsed with the least relative priority levels were the age   range served, youth and family engagement in the services, and peer support. The Youth-Focused Service Accessibility segment prioritised services that were sensitive to the cultural and trauma background of youth. There was some endorsement of the options to have culturally-based services available for cultures common to the local area, or having the option to request a service provider with a certain cultural background. For the Core Health Services attribute, the level that provided mental health and substance misuse counseling, medication management, and physical/sexual health services received the highest utility value. Short Wait Times were also strongly preferred in this segment; Youth-Focused Service Accessibility participants felt that youth should be able to see a counselor either immediately, or after about 72 h. Finally, 24/7 E-Health Services that supplemented inperson services were strongly preferred, as was the option to schedule or reschedule appointments via email, text, or online.

Segment 2: service options
Segment 2, which is comprised of 107 (27.6%) participants, was labelled the Service Options segment. The decisions of these participants were driven by servicerelated attributes, including Core Health Services and Other Services. They also prioritized the Information Sharing and Location attributes. The attributes endorsed with the lowest level of relative priority were the time of appointments, peer support, and youth and family engagement in the organization.
The Service Options segment selected levels representing the most diverse services, including mental health and substance misuse counseling, medication management, and physical/sexual health services from the Core Health Services attribute. This segment also valued the more ancillary options presented in the Other Services attributes, such as education, employment, housing, income support, and legal support. Levels that were limited to employment/education or legal support services both were negatively endorsed. In the Information Sharing attribute, this segment preferred to work with youth to decide which information to share with caregivers. There was little endorsement, positive or negative, for the levels in which the service provider decides with information to share with caregiver, or in which information is available to caregivers with youth consent. There was a negative utility value for no information sharing with caregivers. For the Location attribute, this segment preferred that services be delivered in a dedicated mental health setting or in a casual location such as a café or recreation centre, as opposed to a hospital or school setting.

Segment 3: caregiver integration
Segment 3, comprised of 40 (10.3%) participants, consists of participants who preferred the attributes Caregiver Involvement, Information Sharing, E-Health  Services, and Wait Times. This segment was labelled the Caregiver Integration segment. The attributes reflecting the least relative priority were youth and family engagement, the time of appointments, and peer support. Service providers in the Caregiver Integration segment endorsed a high level of Caregiver Involvement, with caregivers involved in decisions regarding youth counseling, participating in family counseling, or receiving separate counseling from youth. Caregivers receiving their own, separate counseling was less important, although still positively endorsed, while the level specifying no caregiver involvement received a large, negative utility value from this segment. This segment also preferred information sharing with caregivers. This included positive utility values for levels specifying that youth and service providers would work together to decide what information to share, that service providers decided what information to share, and that information is available to caregivers with youth consent. The level specifying that there would be no sharing of personal information with caregivers received negative utility values. The Caregiver Integration segment believed that ehealth services should be offered as a supplement to in-person service, and that service users should be able to schedule or reschedule appointments via email, text, or online. Finally, for the Wait Times attribute, service providers preferred rapid access.
Chi-square analyses of demographic and agency characteristic variables indicated that the segments did not differ based on any of these factors (Table 6).

Discussion
This study aimed to understand the perspectives of service-providing stakeholders on the service characteristics most important to include in IYS settings for youth mental health and substance use disorders. Results revealed that participants strongly endorse service models that provide rapid access to the widest variety of culturally sensitive service options, with supplementary ehealth services, in youth-focused community settings with evening and weekend hours. They prefer caregiver involvement in youth services and treatment decisions, and support youth and family engagement. There were three segments of service providers/organization leaders with somewhat different perspectives. The largest segment most strongly preferred youth-oriented accessibility factors. Another segment, representing over a quarter of service providers, emphasized the importance of offering the most diverse possible set of services to meet a wide range of youth needs. A minority of participants fell into a group that valued the involvement of the youth's caregiver as a top relative priority. The segments did not differ based on demographic characteristics.
Obtaining the buy-in of service providers, service providing agencies, and other stakeholders is essential to the successful implementation of complex health interventions [36,37]. Referred to as part of the "inner setting" of the implementation environment in the implementation science literature, service providers and agencies must have positive attitudes toward an intervention and believe it represents an improvement in order to implement an intervention with fidelity [36,37]. A IYS implementation study found that a strong inner setting during the implementation process was important for the successful implementation of the intervention [38]. The current study further demonstrates that service providers and agency leaders across a sample representing diverse service organizations strongly value youthfocused core service components that are foundational to IYS settings, such as rapid access to a variety of health services for a wide range of youth needs. While the degree of prioritization of different service components differed somewhat across latent segments, overall preferences were very similar for most components of care and service providers universally positively endorsed key preferences that define IYS models. IYS models of youth services, then, appear to be a strong fit with the preferences of service providers at youth-serving agencies in the current youth service landscape, in which IYS services are in the early stages of being scaled up, but are not yet universal.
Caregivers of youth have expressed the need to be more involved in their youth's treatment. Service providers as a whole also support caregiver involvement in care. Information sharing with caregivers is a top priority for service providers, while they also endorse involving caregivers in counseling and decision-making. Indeed, caregivers can provide the impetus for youth to seek services and many wish to be closely involved in their care [8,[39][40][41]. It is important for service providers in IYS settings to recognize this and negotiate with the youth and caregiver to ensure that the caregivers are involved in an appropriate manner.
IYS models of service delivery are rapidly expanding around the world, including multiple Canadian models [21]. Reviews have described prevalent characteristics of such models and presented evidence of positive outcomes [15,16]. IYS models strive to integrate many service components and characteristics considered to be youth friendly and consistently define themselves as Additional genders were removed from the analyses due to small cell sizes youth-friendly settings [9]. The goal is to make these service settings as comprehensive as possible, including a broad range of services to meet youth needs across mental health and substance use, but also including attention to important factors such as employment, education, housing, and other social determinants of health. Indeed, given the preferences expressed by service providers, all of these components of care and care settings should continue to be a driving objective of service development. Based on the current findings, youth service developers and funders of such services should prioritize the integration a wide variety of services, including ehealth service components, while ensuring rapid access and developing strong policies around caregiver involvement. In doing so, they will optimize service provider and agency buy-in to their models by building services that reflect these key stakeholder preferences. Youth and caregiver preferences should also be taken into account when identifying priorities for IYS service development [28,29].

Strengths and limitations
The DCE methodology makes it possible to identify and model stakeholder perspectives in a rigorous manner, with a particular strength of identifying relative priorities for certain service characteristics over others. These results can guide system designers as they engage in difficult decision-making processes, when faced with competing priorities and limited resources that require them to focus on the most important service characteristics first. Areas of agreement across segments highlight the highest priority area for service development, while areas of disagreement represent areas of possible design flexibility, as well as focus areas for consensus building to be contextualized based on service user preferences. The study was conducted throughout Ontario, Canada, and reached participants across a diversity of youth service sectors. The results are therefore considered to illustrate broad themes within youth-serving agencies of various kinds across Ontario. However, the recruitment process was limited to one Canadian province and the sample may not have been representative of Ontario youth-serving organizations. Results may therefore not be generalizable outside of Ontario or in organizations with characteristics not represented in the sample. Different subgroups of participants may emerge with a more diverse sample. The current results refer to service provider/organizational leadership preferences only and should be interpreted together with the perspectives of service users [28,29]. It is also important to keep in mind that a DCE item set is not a psychometrically validated assessment tool. The items were developed through a substantial, collaborative process in which stakeholders were engaged; however, it is possible that some priorities of participants were not included in the item set and overlap between items may have affected the relative importance attributed to different attributes and levels. This study does not make it possible to identify the efficacy or effectiveness of any of the service components. Given the complex nature of IYS settings and service pathways, future effectiveness research should examine outcomes of youth within the model as a whole, to supplement outcome research on individual components of care. Future research should also consider effectiveness based on the service components that have been implemented in a given IYS model.

Conclusions
The core characteristics of IYS settings, i.e., rapid access to a wide range of youth-oriented services, are considered strong priorities by service providers and youthserving organizational leaders. These findings confirm that youth-oriented service providers endorse the importance and relevance of the IYS model as a whole. The development, implementation and scale of IYS models will be facilitated by service provider buy-in to the model. Future IYS research and development work should take service provider priorities into account to build the most responsive, comprehensive service models possible to address youth service access barriers and substantially improve the youth mental health, substance use, and wellness support sector. In building such models, hearing the preferences of diverse stakeholders, including service providers, youth, and caregivers, is essential.
Abbreviations IYS: Integrated Youth Services; DCE: Discrete Choice Experiment