Adaptation and validation of the On Your Own Feet – Transition Experiences Scale evaluating transitions to adult services among adolescent mental health service users in Europe

Purpose: Experiences of young people transitioning from Child and Adolescent Mental Health Services (CAMHS) to Adult Mental Health Services (AMHS) have mostly been investigated qualitatively. This study adapts and validates the On Your Own Feet – Transition Experiences Scale (OYOF-TES) in a sample of CAMHS users in Europe and describes young people ’ s and parents ’ experiences with transition and end of care at CAMHS. Methods: The OYOF-TES was adapted to a mental health setting and translated. An End Of Care (OYOF-EOC) version (self-and parent-report) was developed. A total of 457 young people and 383 parents completed an OYOF-TES or OYOF-EOC. Psychometric properties and descriptives are presented. Results: The Cronbach ’ s alphas of the OYOF-TES and OYOF-EOC parent/self-report ranged from 0.92 to 0.94. The two-factor structure was confirmed. The mean overall satisfaction reported by young people was 6.15 (0 – 10; Abbreviations: CAMHS, child and adolescent mental health service(s); AMHS,

SD=2.92) for transition and 7.14 (0-10; SD=2.37) for care ending.However, 26.7%-36.4% of young people were unsatisfied.Discussion: The OYOF-TES and OYOF-EOC can be used reliably in mental healthcare settings to capture young people's and parents' transition experiences.The majority of young people and parents was satisfied with the process of transition and care ending, yet a third of young people had negative experiences.

Implications and contributions
The questionnaires OYOF-TES and OYOF-EOC can be reliably used to capture CAMHS users' experiences with transition and end of care.Respectively, twenty-seven and thirty-six percent of young people have negative experiences with care ending or transition.Our findings provide input to help clinicians improve young people's and parents' experiences.

Introduction
Young people needing continued specialist mental health treatment after reaching the service's upper age limit (usually between the ages of 16 and 19) of their child and adolescent mental healthcare service (CAMHS) need to transition to an adult mental healthcare service (AMHS).To date, experiences of young people with transition from CAMHS to AMHS have mostly been investigated by retrospective (Islam et al., 2016;Perera et al., 2017;Pontoni et al., 2021;Stagi et al., 2015) and qualitative studies (Appleton et al., 2020;Broad et al., 2017;Cleverley et al., 2020a;Hill et al., 2019;Scholz et al., 2019;Wilson et al., 2015).Such reports highlight that most young people and parents feel insufficiently prepared for this transition (Appleton et al., 2020;Cleverley et al., 2020a;Scholz et al., 2019;Wilson et al., 2015) and experience difficulties in the alignment between CAMHS and AMHS (Scholz et al., 2019).
While providing valuable insights into young people's and parents' experiences with transition from CAMHS to AMHS, retrospective and qualitative studies have methodological limitations, such as the limited numbers of young people and parents who can be included.Furthermore, some studies only included young people who transitioned to AMHS (Hill et al., 2019), whereas the majority of young people do not transition to AMHS (Gerritsen et al., 2022b).To our knowledge, no instruments have been developed to quantitatively assess the experiences of young people and parents with transition in a mental healthcare setting.
The current study aims to adapt and validate the On Your Own Feet -Transition Experiences Scale (van Staa and Sattoe, 2014) (OYOF-TES), a validated instrument originally designed for assessing experiences with transition of those with chronic physical conditions, in a sample of child and adolescent mental healthcare service users in Europe (Gerritsen et al., 2021;Singh et al., 2017).Additionally, we developed a version of the instrument to assess experiences of young people whose care ended (the On Your Own Feet -End of Care, OYOF-EOC).We provide psychometric properties of these instruments and descriptive statistics to illustrate young people's and parents' experiences and satisfaction with transition from CAMHS to AMHS.

Study design and participants
The current study is part of the MILESTONE longitudinal cohort study, in which 39 CAMHS from eight European countries participated (Belgium, Croatia, France, Germany, Ireland, Italy, the Netherlands and the United Kingdom).The study design (Singh et al., 2017) and participants (Gerritsen et al., 2021) have been described previously.CAMHS users aged up to one year younger or up to three months older than the upper age limit of their CAMHS between October 2015 and December 2016 were eligible for participation in the study.Eligible young people were regular CAMHS users with a mental disorder, had no indication of intellectual impairment and were (expected to be) able to complete questionnaires (see Fig. 1 for all inclusion criteria).Participants were assessed at baseline and follow-up at 9, 15, and 24 months.
Written informed consent was obtained from all participants.According to national laws and medical ethical committee regulations, country-specific consent procedures were followed for young people, parents and clinicians.The UK National Research Ethics Service Committee West Midlands -South Birmingham (15/WM/0052) and ethics boards in participating countries approved the study protocol (ISRCTN83240263; NCT03013595).

Procedure
At each time point, a two-hour appointment (face-to-face or by phone) was scheduled with young people and their parents for an interview to assess socio-demographic information and the completion of a range of online questionnaires (on the HealthTracker™ platform).Clinical information on the young person was obtained through their clinician or by accessing medical files.
Before each follow-up assessment, the research assistant contacted the young person, parents and the CAMHS clinician, to inquire whether the young person was still at CAMHS, had transitioned to AMHS or another service or was no longer using mental healthcare services at that time.The questionnaires administered at the following assessment were selected based on this information.If the young person had transitioned to AMHS or had moved from CAMHS to another mental health service, the young person and their parents were requested to complete the OYOF-TES (van Staa and Sattoe, 2014) at the first assessment following the end of care at CAMHS.If the young person had left CAMHS and was no longer using mental health services, the young person and their parents were requested to complete the OYOF-EOC.

Measures
A complete overview of the instruments used in the MILESTONE cohort study at each time-point was provided elsewhere (Gerritsen et al., 2021).More detailed information on the instruments used in this study is provided in the supplementary materials.

On Your Own Feet Transition Experiences Scale (OYOF-TES)
The original version of the OYOF-TES (originally the On Your Own Feet Transfer Experiences Scale) was developed in Dutch and validated by Van Staa and Sattoe (2014) in a sample of 1039 adolescents who had undergone treatment in the Erasmus Medical Centre for a broad range of chronic physical health problems.The parent-and self-report version consisted of 18 items rated on a five-point Likert scale (ranging from 'strongly disagree' (1) to 'strongly agree' (5)) divided into an 'alignment and collaboration'-scale and a 'preparation'-scale.Overall satisfaction with transition was determined with the question 'How satisfied are you with the overall process of your transfer to adult care on a scale ranging from 0 to 10?'.
In this study, the OYOF-TES was adapted for administration in a mental healthcare setting (see Supplementary Table 1).An 'end-of-care' version of the OYOF was developed to assess the experiences with care S.E.Gerritsen et al. (caption on next page) S.E.Gerritsen et al. ending of young people who did not transition to AMHS (OYOF-EOC; see Supplementary Table 2).

Socio-demographic characteristics
Information on gender was registered during recruitment.The highest level of parental education was assessed in the sociodemographic interview and categorized as (1) primary or secondary/ vocational or (2) university.

Clinical information sample
At baseline, clinical classifications as registered in the young person's medical file were provided by clinicians and collapsed to three broad categories: emotional disorders; behavioural/ neurodevelopmental disorders and severe mental illnesses.
At each time-point, self-reported internalising and externalising problems in the past six months were assessed with the Youth Self-Report (Achenbach and Rescorla, 2001) or Adult Self-Report (Achenbach and Rescorla, 2003) scale of the Achenbach System of Empirically Based Assessment (YSR if <18 years; ASR if ≥ 18 years).

Statistical analyses
A non-response analysis was conducted by comparing young people who completed an OYOF-TES or OYOF-EOC with young people who left CAMHS and had not completed the OYOF-TES or OYOF-EOC, based on baseline gender, parental educational level, clinical classifications, and self-reported emotional/behavioural problems using Chi-square tests (categorical variables) and t-tests (continuous variables).
To assess the reliability of the OYOF-TES and the OYOF-EOC (both self-report and parent-report) Cronbach's alphas were calculated and presented.To describe construct validity, minimum residual, and confirmatory factor analyses (CFA) were applied.The CFA was carried out using the 'sem' package in R.This package was used to specify a path diagram describing the relationships between the items and the factors.The summary statistics were then evaluated and the model fit was determined.
Descriptive statistics were used to describe the study sample, young people's and parents' experiences with alignment between CAMHS and AMHS, preparation for transition, appropriateness of end of care, preparation for end of care, and overall satisfaction with either the transition to AMHS or end of care.In addition, sensitivity analyses were conducted to address validity with respect to gender, specific groups of mental health disorders (i.e., severe mental illness, emotional disorder, behavioural/neurodevelopmental disorder), and levels of emotional and behavioural problems.
To investigate which specific experiences contributed to overall satisfaction, bivariate correlations (Pearson's r) between overall satisfaction and the individual items were calculated (Mukaka, 2012) (see Supplementary Materials).
All statistical analyses were performed in R version 3.6.3(R Core Team, 2020) using observed data only.

Development and adaptation of the OYOF-TES and OYOF-EOC (selfreport and parent-report)
The self-report and parent-report versions of the OYOF-TES and OYOF-EOC are included in Supplementary Tables 1 and 2. Minor amendments were made to the wording of the items of the original OYOF-TES (van Staa and Sattoe, 2014) and several items were added to adapt the instruments for administration in a mental healthcare setting based on feedback from transition experts, including young people with experience in mental health services transitions, and the NICE guidelines on transition (Singh et al., 2016).
To assess the experiences with ending care in CAMHS of young people who did not transition to AMHS an additional version of the OYOF was developed: an 'end-of-care' version (OYOF-EOC), assessing appropriateness of end of care (subscale 1) and preparation for end of care (subscale 2).
The process of development and adaptation of these measures is described in more detail in the supplementary materials.

Sample
A total of 763 participants participated in a baseline assessment (see Fig. 1 for the flowchart of the study and Supplementary Table 3).A complete description of the demographic and clinical characteristics of the MILESTONE cohort is provided elsewhere (Gerritsen et al., 2021).Over a period of two years and three follow-up assessments 165 (21.6%) young people completed the OYOF-TES and 292 (38.3%) young people completed the OYOF-EOC.A total of 306 (40.1%) young people did not complete an OYOF-TES or OYOF-EOC.For 14.7% of these 306 (n = 43), this was due to them staying in CAMHS.We compared the 457 young people that completed the OYOF-TES (n = 165) or OYOF-EOC (n = 292) to the 263 young people who left CAMHS but did not complete either OYOF-scale (see Table 1).Young people who completed an OYOF-scale were less often classified with a 'severe mental illness' (χ2=11.924,df=1, p < .001)than those who did not complete any OYOF-scales, no other differences were found.
A total of 140 (18.3%)parents completed an OYOF-TES and 243 Note.Not all participating young people, parents/carers and clinicians completed (all) measures at the different time-points.See Table 1 for proportions of missing data per measure. 1 The upper age limit of the participating CAMHS was 18 years for two-thirds of services, or applied flexibly, varying between 16 and 19 years of age.
2 Young people who stayed in CAMHS were not administered an OYOF-TES or OYOF-EOC. 3Without completing an OYOF-TES or OYOF-EOC.6 young people withdrew after completing an OYOF-TES or OYOF-EOC.The data of these 6 young people was included in this study.(31.8%) completed an OYOF-EOC.Twenty-five parents completed an OYOF-TES or an OYOF-EOC although their child did not.

Construct validity
A minimum residual factor-analysis was applied on all versions of the OYOF-TES to assess whether the two-factor structure established by Van Staa and Sattoe (2014) applied to the MILESTONE data as well.Inspection of scree plots indicated that a two-factor structure was optimal.Additionally, a CFA was conducted.A two-factor structure with the factors 'alignment' and 'preparation' for the OYOF-TES versions and the factors 'appropriateness' and 'preparation' for the OYOF-EOC versions was confirmed.The factor loadings derived from the CFA are presented in Tables 2 and 3 and supplementary tables 1 and 2 for each version separately.

Young people's and parents' satisfaction and experiences with the transition to AMHS
Measuring satisfaction with healthcare is acknowledged to be complex (Crow et al., 2002) with some researchers viewing a score of 3 or less on a 10-point scale as indicative of dissatisfaction whereas others would utilize the halfway point (score of 5) as the demarcation between satisfaction or dissatisfaction.In this paper, scores of 6 and above were interpreted as indicating satisfaction and those below 6 indicating dissatisfaction.Fig. 2a shows the distribution of overall satisfaction scores with transition to AMHS of young people and parents.The mean overall transition satisfaction score reported by young people was 6.15 (range 0-10; SD=2.92) and 6.13 (SD=2.74)reported by parents.More than one in three young people and parents were unsatisfied with their transition: 60 (36.4%) young people and 53 (37.9%) parents rated the transition less than a '6′.Similar proportions of young people (40.0%, n = 66) and parents (37.9%, n = 53) were very satisfied (they rated their transition as an '8' or better).The sensitivity analyses to address validity of the overall mean transition satisfaction score with respect to gender, specific groups of mental health disorders, and levels of emotional and behavioural problems are presented in Supplementary Table 4.
The responses to the different items of the OYOF-TES, reflecting the young person's experiences with transition, are described in Table 2.The correlations with overall transition satisfaction are presented per item as well.Young people's responses to the statements measuring different aspects of alignment between CAMHS and AMHS are mixed.Overall, young people seemed positive regarding the care they received at AMHS.Young people were less positive regarding alignment between CAMHS and AMHS: only 30-40% reported having experienced good collaboration, similarities in the way of working, and knowing what to expect at AMHS.With regard to preparation, experiences were mixed: roughly two in three young people stated the transition announcement was well-timed and did not come as a surprise, whereas only half felt the timing of transition was right, that they were well prepared, and that they had a say in whether they would transition and when.
The responses of parents, presented in Supplementary Table 1, were similar to the responses of young people.However, most parents did not feel prepared for how their role, responsibility, and involvement would change.

Young people's and parents' satisfaction and experiences with the end of care
Young people were quite satisfied with their care ending (mean overall satisfaction score=6.92,range 0-10; SD=2.64), as were their parents (mean satisfaction score=6.87;SD=2.87; see Fig. 2b).A total of 78 (26.7%) young people and 71 parents (29.2%) were unsatisfied (scored below 6).About half of the young people (n = 150, 51.4%) and parents (n = 128, 52.7%) were very satisfied (scored ≥8).The sensitivity analyses to address validity of the overall mean end of care satisfaction score with respect to gender, specific groups of mental health disorders, and levels of emotional and behavioural problems are presented in Supplementary Table 4.
Table 3 describes young people's experiences with end of care.Most young people indicated they knew how to manage on their own and where to go for help if needed.Similar to the responses to the OYOF-TES, two in three young people and parents who completed the OYOF-EOC stated they were ready to end their care at CAMHS.Young people who completed the OYOF-EOC indicated that the timing of end of care was right, they felt prepared, and it was announced in a timely manner.Furthermore, almost two in three young people indicated having had a say in whether and when their care would end.
The responses of parents to the OYOF-EOC (see Supplementary Table 2) were similar to the responses of young people, with the exception that most parents felt they were not being involved in the decision whether the care would end or not.

The relationship between transition experiences and overall satisfaction
All correlations between transition experiences and overall satisfaction were significant at p<.001.Several items had a moderate to strong association (r≥.6) with young people's overall satisfaction (see Tables 2 and 3 and supplementary tables 1 and 2): the new care provider being informed about the young person and their condition, having been taken good care of in the adult setting, good collaboration between CAMHS and AMHS, having been well prepared for the transition to AMHS, and having received enough information about the transition.For parents, confidence in the young person's AMHS team and happiness with received care were important, as was the readiness and preparedness to transition, whether they received enough information about the transition, their preparedness for how their role would change and whether they were still informed about important decisions regarding their child's condition and treatment at AMHS.
The experiences with the care at CAMHS ending that were associated most strongly with the overall satisfaction of young people were: having learned helpful things in CAMHS, the young person's parents' readiness for care to end according to the young person (the correlation with their own readiness was lower), the timing of the care ending, having been prepared for their care to end, having received enough information, and having had a say in when the care would end.
For parents, almost all the experiences relating to the preparation for the end of care were significantly associated with their satisfaction.Additionally, parents who were glad their child did not have to go to AMHS were more satisfied.

Discussion
This study describes the adaptation, translation and initial validation of the OYOF-TES and OYOF-EOC within a sample of CAMHS users in Europe.The reliability and construct validity of all measures were good and comparable to the original measure (van Staa and Sattoe, 2014).The mean satisfaction with transition reported by young people in this study was also comparable to the satisfaction with transition in a population of adolescents with chronic physical conditions (van Staa and Sattoe, 2014).Additionally, this study describes young people's and parents' transition experiences.Thirty-six percent of young people were unsatisfied with transition, whereas 27% of young people were unsatisfied with care ending.Parents were slightly more unsatisfied, but overall satisfaction of young people and parents were comparable.

Experiences with end of care at CAMHS
Our finding that young people were relatively satisfied with care at CAMHS ending, and more satisfied overall than young people who transitioned to AMHS, was unexpected considering the scientific literature suggesting many young people 'fall in a [care] gap' (Appleton et al., 2019).However, it is in line with more recent findings that most 0.90, 'moderate' between 0.50 and 0.70, 'low' between 0.30 to 0.50 and 'negligible' between 0.00 to 0.30 (23).young people who leave CAMHS, and do not transition to AMHS, have relatively good outcomes (Gerritsen et al., 2022b).This may be because CAMHS users who experience an end of care, have relatively few mental health problems as they reach the upper age limit of their CAMHS and often do not need continued treatment (Gerritsen et al., 2021(Gerritsen et al., , 2022b)).This group is likely to be satisfied with care at CAMHS ending and may be overrepresented in the MILESTONE cohort (see 'strengths and limitations'), which may have resulted in an overestimation of the satisfaction with end of care.Problems with transition may also remain hidden by CAMHS clinicians' willingness to continue treatment of CAMHS users beyond the upper age limit, considering the lack of capacity and waiting lists at AMHS.

Experiences with transition or end of care in relation to satisfaction
The majority of young people and parents rated the process to transition or end care as 'satisfactory' (indicated by a score of 6 or higher).Nevertheless, 27%− 36% of young people were unsatisfied with the way in which care ended at CAMHS or the transition to AMHS.Our findings are not completely in line with findings from qualitative studies (Appleton et al., 2020;Broad et al., 2017;Cleverley et al., 2020a;Hill et al., 2019;Scholz et al., 2019;Wilson et al., 2015), which describe young people's and parents' experiences that are more negative.An analysis of young people's and parents' experiences on an item-level provides potential explanations for the discrepancies between our findings and the findings from previous studies, but also reveals some contradictory findings.
Previous studies (Cleverley et al., 2020b;Gerritsen et al., 2022a) emphasize the importance of involving young people and parents in the decision-making process regarding transition.In our study, having had a say in whether, when or where they would transition had a low correlation with young people's satisfaction, suggesting this may not be as important overall.Previous research has also shown that the timing of transition can feel abrupt and arbitrary (Broad et al., 2017), yet two in three young people in our study reported that transition was announced timely and did not come as a surprise.In line with findings from previous qualitative studies (Cleverley et al., 2020a), the importance of being prepared and receiving enough information was reflected in the moderate correlation with overall satisfaction.
Roughly one in three young people felt their new care providers were not well informed and this seemed to affect their overall satisfaction of transition as well, which is in line with previous research indicating young people and parents experience a lack of coordination and communication between CAMHS and AMHS (Scholz et al., 2019).Only 40-44% of young people and parents indicated that CAMHS and AMHS collaborated well, which has also been reported in previous qualitative research (Scholz et al., 2019).
Approximately one in three of young people whose care ended expressed regret with ending specialist mental healthcare, whereas actively having a say in continuation of care significantly impacted young people's satisfaction with care ending (more so than for those who transitioned).Considering that young people who report a need for ongoing treatment as they reach the upper age limit of their CAMHS are more likely to experience increasing mental health problems in the transition period (Gerritsen et al., 2022b), this is a worrying finding.Fortunately, most young people whose care ended indicated they did know where to go in case they needed help.
Finally, our study confirms previous findings that parental involvement in treatment decisions is much less common in AMHS than in CAMHS (Hill et al., 2019), yet the low correlation with transition satisfaction suggests this may not lead to a lot of frustration and powerlessness for most parents.Perhaps some experience a sense of relief as their responsibility decreases (Hill et al., 2019).

Strengths and limitations
The current study has several strengths: the quantitative approach allowed us to describe the experiences of a sample of 457 young people in CAMHS who transitioned to AMHS or whose care ended as they reached their service's upper age limit.Because young people were recruited in eight countries, our findings are not limited to a mental health system specific to one country.
However, a bias may have resulted in an overestimation of the satisfaction with transition: selection bias potentially affects the representativeness of the cohort (discussed in more detail elsewhere, Gerritsen et al., 2022a), it is not unlikely that participation in the MILESTONE study may have sensitised CAMHS clinicians to several aspects of transition (i.e.informing young people on transition), and non-response analyses showed that young people who completed an OYOF-TES or OYOF-EOC were less likely to have a clinical classification of a severe mental illness.Our findings therefore need to be interpreted with caution and require replication before informing transition policy.
Because of the proportion of missing data on the OYOF-TES and other clinical and service-use characteristics, we could not study associations between these characteristics and young people's experiences.This should be considered a limitation as well, and offers a suggestion for further research within this population.
Lastly, the present study was conducted as part of the MILESTONE cohort study.In this context, we were able to test the applicability of the OYOF-TES and -EOC.However, because the study was not designed as a validation study of the OYOF, we were unable to test other aspects of validity, such as test-retest, convergent, and discriminant validity.

Conclusions and recommendations
Our study shows that the OYOF-TES and OYOF-EOC can be reliably used in mental healthcare settings to capture transition experiences of young people and their parents and provides clinicians and managers with an easy to use questionnaire to evaluate transitional care in their mental health services.The majority of young people and parents was satisfied with the process of transition and care ending, yet a third of young people had negative experiences.Our findings suggest that adequate preparation for transition or discharge, informing the new care provider, preparing parents for their changing role, increasing independence of young people and, in general, improving collaboration between CAMHS and AMHS may improve young people's experiences with transition.Although replication of our findings is important, they provide insight into important aspects of how to improve transitional care for young people and their parents.

Fig. 1 .
Fig. 1.CONSORT Flow Diagram of participants.Note.Not all participating young people, parents/carers and clinicians completed (all) measures at the different time-points.See Table1for proportions of missing data per measure.1The upper age limit of the participating CAMHS was 18 years for two-thirds of services, or applied flexibly, varying between 16 and 19 years of age. 2 Young people who stayed in CAMHS were not administered an OYOF-TES or OYOF-EOC.3Without completing an OYOF-TES or OYOF-EOC.6 young people withdrew after completing an OYOF-TES or OYOF-EOC.The data of these 6 young people was

Fig. 2 .
Fig. 2. Young People's and Parents' Overall Satisfaction with Transition to AMHS and End of Care.

Table 1
Baseline characteristics of young people who left CAMHS and completed an OYOF-TES or OYOF-EOC and young people who had not completed either.
* Young people who stayed in CAMHS and therefore did not complete an OYOF-TES or OYOF-EOC were not included.S.E.Gerritsen et al.