The development and psychometric validation of a survey to measure the subjective well-being of care leavers

Young people who age out of state care are at risk of a range of negative outcomes. In England, national data provides only five indicators of care leavers ’ lives and there are no measures of how young people themselves feel about their transition to adulthood. To fill this gap a new survey to measure subjective wellbeing was co-produced with 31 care leavers. The survey was then distributed by 21 local authorities and completed by 1804 care leavers. The responses revealed a steep decline in wellbeing after leaving care, a wide variation in care leavers ’ wellbeing depending on the local authority responsible for their care, and that some groups, such as those with a disability, were more vulnerable to low wellbeing. The survey was also validated using psychometric analyses. Latent factors were extracted, dimensionality tested and differential item functioning (DIF) was used to see if different groups of care leavers responded similarly to questions. The association between the total survey score and the commonly used Office for National Statistics four personal wellbeing questions was examined. The survey had good reliability across each of the statistics but data loaded onto a five-factor solution rather than the theorised four. DIF analysis found differences by sex, ethnicity and disability. Overall, the survey was found to be a valid and reliable measure of care leavers ’ subjective wellbeing providing practitioners with information on which aspects of life were going well and where practice and policy needed to change.


Introduction
Every year about 12,300 young people aged 16-18 leave state care in England having been looked after in foster or residential care.They are eligible for care leaver support up to the age of 25 if they were in care between their 16th and 18th birthday (Children & Social Work Act 2017, s3).A detailed pathway plan is begun at about 16 years of age and the plan sets out the support that will be provided.The plan is reviewed regularly and includes support from a personal advisor, decisions on where the young person will live and financial help with living costs.Although local authorities (LAs) have often been caring for young people as their corporate parent for many years and have a duty to provide support, there is very little information published at a national level on how young people fare as care leavers.There are only five national outcome indicators published annually for care leavers aged 17-21 years old (Department for Education, 2023).These outcome indicators are (a) whether the LA thinks the young person's accommodation is suitable, (b) the type of accommodation, (c) activity (whether the care leaver is employed, in training or reasons why not) (d) whether the care leaver has continued to live with their previous foster carer and (e) the percentage of care leavers remaining in touch with their LA.Positively the majority of care leavers (about 90 %) are recorded as being in contact with their LA but no data are collected nationally on young people's views on whether they feel that their transition to adulthood has gone well.All the government's outcome indicators are based on judgements made by professionals.

Background
Previous international (e.g., Mann-Feder & Goyette, 2019;Gabriel et al., 2021) and UK research (e.g., Butterworth et al., 2017, Stein, 2012;Munro & Simkiss, 2020;Dixon, 2023) have highlighted how difficult the transition from care to independence can be for care leavers.In England, studies have found that care leavers are at greater risk of criminal convictions (Office for National Statistics, 2022), homelessness (APPG, 2017), mental health problems (Department of Health, 2012; Barnardo's, 2017), teenage pregnancy (Craine et al., 2014) and premature death (Murray et al., 2020).However, there are also many accounts of care leavers who lead happy fulfilled lives, achieve academically (e.g., Aspire To More, n.d.) and are talented artists, writers and poets (e.g., Which One Grew up in Care?They All Did! (2018).Recent research has also highlighted how some transitions (e.g., entry to higher education) are more often delayed for care leavers compared with the general population and therefore the often quoted low rates of entry to higher education for care leavers are inaccurate (Harrison, 2020).
The focus on 'poor outcomes' hides the heterogeneity of the care leaver population preventing a greater understanding of where interventions would be best placed and does not allow the voice of care leavers to influence policy and practice.Care leavers are not asked systematically how they think they are doing, what has worked well and what might improve their experience of leaving care.This article reports on how a survey to measure the subjective wellbeing of care leavers was co-produced and its later psychometric validation.There are many definitions of wellbeing in the literature.In our research, we have defined wellbeing as 'feeling good and doing well at an individual and interpersonal level' to reflect the multidimensional nature of wellbeing (Keyes & Annas, 2009).

Measuring wellbeing
Internationally, there has been growing interest in measuring the population's wellbeing, understanding how it changes across the life course and whether countries' wellbeing can be compared (e.g., Ruggeri et al., 2020;Wallace et al., 2020).Objective wellbeing has been measured for many years on specific areas of life such as life expectancy while more recently frameworks have been created to collect and combine objective measures on many different areas of life (e.g., The Organisation for Economic Co-operation and Development, 2020).Increasingly frameworks also include subjective measures that ask how a person thinks and feels about their own life (e.g., Das et al., 2020).Subjective measures are recognised as an important aspect of understanding an individual's wellbeing and, it is argued, are the best proxy for overall wellbeing (Pavot, 2018).The use of subjective measures has led to important contributions to understanding physical, psychological and social health (e.g., Finkelstein et al., 2012;Lee et al., 2013;O'Donnel et al., 2014;Ruggeri et al., 2020).Concerns remain about the conceptual framework and measurement of subjective wellbeing (e.g., Krueger & Stone, 2014;Ruggeri et al., 2020) but there is a general acceptance that the measurement of wellbeing needs a multidimensional approach that includes subjective measures.
In the UK, an updated framework for measuring national wellbeing was introduced in 2023 (ONS, 2023a).The ONS framework is made up of 60 indicators informed by ten domains with data collected from a range of sources.The domains are 'personal wellbeing'; 'our relationships'; 'health'; 'what we do'; 'where we live'; 'personal finance'; 'education and skills'; 'economy'; 'governance'; and 'environment'.The framework includes four subjective wellbeing questions that have been used to report the general population's subjective wellbeing since 2010.They are commonly known as the 'ONS 4' and the questions ask: The ONS 4 have become a harmonised standard for measuring subjective wellbeing and are used in many different national surveys (ONS, 2018a).Each of the four questions is on a zero to ten scale (with zero being very low and 10 being very high).
While the new framework improves the range of wellbeing data available on the older adult population, data remain very limited on young adults (under 25 years) and on vulnerable groups such as those with a disability or those who have been maltreated.

Measuring care leavers' subjective wellbeing
Previous studies (e.g., Sulimani-Aidan et al., 2022;Refaeli et al., 2019;Sims-Schouten & Hayden, 2017) examining the wellbeing of care leavers have provided important information on single aspects of care leavers' subjective wellbeing such as their mental health or their views on the availability of support services.However, research has not considered all the different domains that contribute to wellbeing or how the indicators that are important for care leavers might differ from those of the general population.
Our previous research investigated the subjective wellbeing of children (age 4-17 years) in care (authors).Using focus groups attended by 140 children, we found that children in care did agree with many of the national wellbeing child indicators (ONS, 2018b) but also identified many different ones.For example, the frequency of contact with birth families was important for children in care and this indicator of relationships was not included for the general population.The research led to the development of an online survey for children in care (authors) and the Bright Spots Programme where LAs who had used the survey could share good practice.Therefore, when we considered the wellbeing of care leavers (aged 18-25 years old) we expected that they too would identify different indicators to those in the national measuring wellbeing framework.
Research funding from the Hadley Trust enabled the methodology that had been used to create the children in care survey to be replicated and applied with care leavers (authors).The study had two aims.The first was to co-produce with care leavers a survey to measure their subjective wellbeing and the second was to consider the survey's reliability and validity once in use.The study had ethical approval from the University of Bristol's ethical committee.

Methodology
The description of the method is divided into three sections.First the development of the survey, the second section describes the sampling procedure and data collection and third the method for the item response theory (IRT) validation of the survey questions.

Development of the survey
Work began in 2017 to develop the new survey in partnership with Coram Voice (a children's rights charity).First, a literature review of care leavers' views on their transition from care was compiled to identify common themes (authors).National and international wellbeing surveys were also searched for questions that had been tested with young people that could be incorporated into the new survey to provide comparative data on young people in the general population.
To co-produce the new survey, two local authorities (LAs): one large metropolitan authority and another smaller rural area agreed to participate.Local Authorities are responsible for services (e.g., social work, transport, education) within a geographical area The LAs who are responsible for Children's Services (employing social workers and personal advisors) sent information to their care leavers about the aims of the development work inviting participation in focus groups.Thirty-one care leavers gave their consent and a £20 voucher was offered as an acknowledgement of the time and expertise care leavers were providing.Over a year, four workshops were held in each LA with food and drinks provided.Care leavers in the initial workshops were asked, 'What makes a good life?' -this question produced a long list of items that they considered important.There were areas that they held in common with peers in the general population, but others were highlighted that were unique to their care experiences.For example, they wanted a full explanation as to why they had been in care and to be involved in their transition planning.They were also concerned about the relationships they had with their workers and wanted someone they could trust and who was easy to contact.Further workshops with care leavers reduced the 'long list' and care leavers were asked about the inclusion of questions that had been identified during the literature search.From the ONS framework, six questions were agreed as very important for care leavers: a question on the presence of disability and long-term health conditions, a question asking about being a victim of crime and the four personal ONS wellbeing scales.The responses on the first three wellbeing scales were grouped into the categories of low (0-4), medium (5-6), high (7-8), or very high (9-10).The responses to the question on anxiety were grouped into very low anxiety (0-1), low (2-3), moderate (4-5), and high anxiety (6-10).These bands are the same as those used by the ONS to describe the wellbeing of the general population.A brief measure of stress was selected (PSS-4; Cohen et al., 1983) as care leavers stated that the transition from care was often stressful.
Thirteen questions were also taken from our children in care survey, as they were relevant for care leavers and provided an opportunity to compare the responses of care leavers with children in care.The wording was slightly changed e.g., asking about ease of contact with their personal advisors rather than with their social worker.Finally, we added some questions on the presence of positive and negative emotions and some open text boxes for comments.
The questions (indicators) were placed into four domains based on discussions with the care leavers and where the researchers thought they might theoretically belong.The domains were, 'People in your life', 'Being a care leaver', 'Living independently', and 'Feelings'.However, where an indicator belonged was not clear-cut, and some could have gone into more than one domain (Fig. 1).All the survey questions were optional, completed anonymously and the survey opened with a requirement for consent otherwise the survey closed.
The draft survey was piloted in six LAs and completed by 420 care leavers.In addition, 16 cognitive interviews were completed that aimed to further understand how care leavers were processing, and responding to the survey questions.For example, a question from the ONS wellbeing framework that asked, 'In the last year have you been a victim of crime?' was not well understood.Care leavers wondered if to be a 'victim' the crime had to have been reported to the police, and some were unsure if the question was asking if they had committed the crime.This question was removed from the survey and further changes were made to response options and the survey was then launched.
Between 2019 and 2020, 21 LAs commissioned the survey and distributed it to their care leavers aged 18-25 years.Response rates were generally good with an average of 39 % with some LAs achieving a 74 % response rate.Each LA was provided with a report based on the responses from their care leavers.

The sample (n = 1804)
The 1804 care leavers who completed the survey ranged in age from 18 to 25 years: 793 (45 %) were male, 862 (49 %) were female, and 105 (6 %) did not respond to this question.The majority (59 %) described themselves as being of White ethnicity, 33 % self-identified as being of Asian, Black, Mixed or of Other ethnicities and 8 % chose not to respond to the question.More than one in five (22 %) recorded that they had a disability or limiting long-term health condition: a larger proportion compared with the 16 % of young people aged 16-24 years in the general population who responded to this question in the national census (ONS, 2023b).
Comparing the national data (Department for Education, 2019) on care leavers aged 19-21 years with care leavers of the same age who completed the surveys, the sample was broadly representative although young women were over-represented (Table 1).
The sample size of more than a thousand provided the opportunity to further test the reliability and validity of the survey questions.

Method for testing the reliability and validity of the survey
The structure and psychometric properties of the survey were investigated in stages using Item Response Theory (IRT).The psychometric analysis aimed to first extract latent factors and test the dimensionality.Second, to examine, using differential item functioning (DIF) analyses, whether the questions functioned differently for different groups of care leavers.For example, to examine whether the questions were easier or more difficult to endorse for males or females even after accounting for their responses across all questions.
Third, use the ONS 4 to evaluate the convergent and divergent validity of the survey scores for each of the four domains and the total survey score.One of IRT's biggest advantages is that it can be used to determine how suitable items are to measure the latent traits, so it can increase reliable information and validity of the scale as a whole (Oishi, 2007;Nima et al., 2020).
First data were entered into the R statistical software for initial cleaning and descriptive analysis resulting in 1760 survey questionnaires being available for the psychometric analyses.The survey questions have mixed response formats, including dichotomous (yes/no) responses, option lists, and various 3, 5 and 11-point rating scales, as well as some open-ended response questions.
All items (except the open-ended questions) were scored such that higher scores reflected more positive outcomes for the care leaver, except for the question that asked whether the young person was pregnant or had a child where a negative response was a positive outcome.Apart from reverse scoring where appropriate, the scoring structure of the original response scales was maintained.The two exceptions to this were the 'Support' indicator, which collapsed to a maximum score of 4, and the 'Stress' indicator, which was dichotomised, to prevent them from dominating their respective domains.A composite score was then created by summing the responses to the 28 questions from the four domains.
Both the Exploratory Factor analysis (EFA) and the Partial Credit Model (PCM) were applied using the TAM package (Robitzsch et al., 2022).The PCM when applied to survey data expects that the probability of a person responding in a particular category on each of the questions depends on their level of the construct (e.g., their overall wellbeing as a care leaver) and the relative level of the construct reflected by each of the response categories for the question.PCM is in the family of Rasch models and is appropriate for the mixed dichotomously scored and rating scale data that make up the survey.As a Rasch model, when data fit the model, the total score is a sufficient statistic, i.e., it summarises all the information in the response patterns, which is important if the total score(s) of the survey is used in future to make judgements.The EFA and the PCM were applied to the whole dataset.Item difficulty evaluation and tests of dimensionality were also applied.
In the next section, the main results of the survey are first described followed by the results of the psychometric analysis.

Results from the survey
The analysis of the survey responses provided three key findings.First, we found that the majority of care leavers rated themselves as having moderate to high wellbeing-perhaps a surprising result.However, a larger proportion also rated themselves as having low wellbeing compared with their peers in the general population (Table 2).
Just over a quarter of care leavers reported low life satisfaction compared with only 3 % of their peers in the general population and about one in three (34 %) reported high anxiety compared with about one in four (20 %) young adults in the general population (ONS, 2019).
The factors from the four domains associated with low and very high wellbeing for care leavers are set out in the online open access report [authors' own] and shown graphically in the Supplementary material.
While lower care leaver wellbeing replicates the findings in previous research, our analysis was able to provide more nuanced findings.Care leavers who recorded that they had a disability or limiting long-term health condition were particularly vulnerable to low wellbeing.They felt lonelier, fewer felt safe and settled where they lived, struggled financially and less frequently reported that they had goals and plans for the future compared with the responses from other care leavers.
While research has found that in the general population, wellbeing decreases from adolescence until middle age (ONS, 2019), the decline was much steeper for care leavers).For example, there was a 4 % decrease in very high life satisfaction from the teenage years to young adulthood in the general population but for care leavers, the decrease  was 15 %.The second key finding was the steep decline in wellbeing after leaving care (Table 2).The children's survey uses three of the ONS 4 questions: life satisfaction, happiness yesterday, and whether the things done in life are worthwhile as part of their survey.The question on anxiety is not asked.Table 3 shows the percentage scoring low on each of the scales (0-4) and very high (score of 9-10).
Third, the local authority variation in the wellbeing of their care leavers was very marked.While the average low wellbeing of care leavers was 30 % across all the 21 LAs it varied by LA between 14 % and 44 % of their care leavers.The disparity in the quality of support provided by local authorities is a widely acknowledged, longstanding issue.It was evident in this survey.In one LA 50 % of their care leavers did not always feel safe in their home, compared with another LA where the same was true for 20 %.
The LAs who had commissioned the survey acted upon their own findings in different ways.For example, East Riding LA set up a football team to tackle loneliness; Stockport LA piloted video doorbells to reduce feeling unsafe at home and Sheffield held a positive body image fashion show as so many young people were unhappy with their appearance.More than a hundred practice changes and innovations have been collated and are available at https://coramvoice.org.uk/for-professionals/bright-spots/resource-bank/.

Results from the item response theory analyses
The survey questions had theoretically been allocated to four domains ('People in your life', 'Being a care leaver', 'Living independently', and 'Feelings').First, we were interested in whether the four domains were the best fit for the data.An exploratory factor analysis (EFA) was applied to the survey using marginal maximum likelihood estimation, which compared the relative fit of 2 to 6-factor solutions.These five models were compared using the Bayesian Information Criterion (BIC), which showed that a 5-factor solution was the best fit for the data rather than four.An oblique, oblimin rotation was applied to the best-fitting factor solution to obtain the standardized loadings (Table 3).
Reliability is a generic term used to indicate the proportion of observed variance attributable to construct-relevant variance, or what is sometimes referred to as true variance.It was first evaluated with Cronbach's alpha coefficient (0.79), which indicated an acceptable to a good level of reliability for the survey.Given the well-known deficiencies in alpha as a measure of reliability (Cronbach & Shavelson, 2004), a Schmid-Leiman transformation was applied to the best-fitting factor solution to further fit a bifactor model to obtain McDonald's omega hierarchical (ω h = 0.64) and total (ω t = 0.93) reliability coefficients.The omega hierarchical also indicates how strongly the question responses are associated with a single, general factor.Moreover, the fact that 64 % of the variance of the composite score from the survey could be attributed to variance on a general factor justified the subsequent application of the unidimensional Partial Credit Model to the response data.
As can be seen in Table 3, the factor analysis identified that the questions did not always load as expected.The 'managing financially' and the 'understanding of why in care' questions did not load onto any of the five empirical factors.Whether or not the care leaver was a parent loaded onto a different factor than the expected 'People in your life' domain while the question asking if the care leaver had a pet showed cross-loadings across factors.The questions that comprised the 'Living Independently' domain split in their loadings across two factors, thus explaining why the best fitting solution was 5 factors rather than the theorised four.The five factors identified at this point provided a hypothetical structure for the use of a total scale.Further analyses followed to assess the dimensionality and validity of the survey, particularly whether in future a total score could be used.

Partial credit model fit
Under the PCM, the survey showed a similarly good level of reliability (WLE reliability = 0.78).However, the responses showed an overall misfit to the PCM; total model max χ2(378) = 491.88,p <.001; SRMR = 0.09.The item-level statistics (see Table 4) showed acceptable fit to the PCM for 26 (96 %) of the 28 items (questions) according to the outfit statistic, and only a single item misfit in terms of both the outfit and infit statistics.
The worst-fitting question asked how many leaving care workers the care leavers had in the previous 12 months (Outfit = 1.53,Infit = 1.32).Fig. 2 shows the 'number of leaving care workers' indicator's deviation from model expectation, and particularly how it under-discriminates across the range of the construct, which is consistent with the high outfit and infit values.

Category functioning
As the survey uses multi-category, Likert-type response scales (e.g., always, most of the time sometimes, hardly ever, never), the PCM provides estimates of the 'thresholds' between each of the adjacent categories, i.e., the level of the construct where a response in either adjacent category is equally likely.If the response scale and its categories are functioning as intended, the ordering of these thresholds should reflect the overall ordering of the construct.Three of the questions (managing financially, happiness with appearance and support) had disordered category threshold estimates relative to the overall ordering (as represented by the total scores).This disordering was most pronounced for the 'managing financially' question where the second response category (finding it quite difficult) was never the most probable response for any level of construct.However, the disordering was reasonably minor and was remedied by combining just two adjacent categories.Consistent with this approach, both the ONS and our reports for LAs combine the response options 'finding it very difficult' and 'finding it quite difficult' into one category.

Differential item functioning (DIF)
To understand whether the questions behaved differently across different groups of care leavers, DIF analyses examined three groupings: sex, ethnicity, and disability (Table 5).Overall, there was significant DIF by sex (χ 2 (56) = 591.42,p <.001), ethnicity (χ 2 (56) = 489.81,p <.001) and disability (χ 2 (56) = 371.61,p <.001) compared to the model that assumed no DIF.As shown in Table 6, the improvement in fit from adding group-specific item parameters to the model improved the fit for both the gender and ethnicity factors, even after penalising for the extra parameters in the model (see the BIC statistic), but not for the disability factor.
The questions that showed the most significant gender DIF were the questions that asked, 'How happy are you with the way you look?' and 'Are you a parent?' Males found it easier to endorse the former, and females the latter, even after controlling for their total scores across all items.The questions that showed the most significant ethnicity and disability DIF were the 'having a pet' and 'support' questions.Care leavers who identified as White ethnicity or having a disability were more likely to indicate they had a pet as well as a greater number of different people supporting them, even after controlling for their total scores relative to non-White and non-disabled respondents.

Test of unidimensionality
The Principal Component Analysis of the model residuals identified two main components that accounted for 19.9 % of the residual variance, which provided some further evidence of violations of unidimensionality across the questions.The first principal component of the model residuals indicated a large negative loading on the question that asks, 'Do you know who your leaving care worker is?' versus positive loadings of a cluster of questions that ask about the support care leavers receive from different people.The second principal component of the model residuals indicated a cluster of positively loading items that ask about support and having a partner versus a cluster of negatively loading items about feeling safe and settled where the young person lived.Fig. 3 Table 4 Question and model statistics for the total survey scale.BIC is a method for scoring and selecting a model.
uses colour-coded questions to illustrate their overall contribution to the combined explained variance of the first two principal components.

Concurrent validity
The person estimates from the PCM for the four domains and the total score of the survey were then correlated with the ONS 4 using the Pearson correlation coefficient (r).Given both are intended as assessments of subjective wellbeing, it was hypothesised that higher scores on the [ name of the survey] and the individual domains would be positively correlated with higher scores on the ONS scales of Happiness Yesterday, Things done in life are worthwhile, and Life Satisfaction and negatively correlated with the anxiety question.As can be seen in Table 6, this pattern played out for the survey total and domain level scales but was most pronounced for the total score and the 'Feelings' domain, which showed moderate to high correlations with the ONS 4. The 'Living Independently' domain showed moderate to low correlations, and the two other domain correlations were low to moderate but

Table 6
Means, standard deviations, and correlations with confidence intervals.in the expected directions.
Overall, the psychometric validation found that the survey had a good level of reliability across each of the statistics.Similarly, it showed good item-level fit to the PCM in both statistical and graphical inspection of the item and category functioning and a good level of concurrent validity.The survey did however show an overall misfit to the PCM, which was consistent with the factor analytic and PCA of residuals that indicated that the survey is not strictly unidimensional and thus the total score should be interpreted with some caution.

Discussion
The research set out to co-produce a survey with care leavers that would measure their subjective wellbeing and validate the survey once in use.The survey was successfully co-produced, and over 1800 care leavers responded when it was distributed by 21 LAs.It has been argued (e.g., Pavot, 2018) that because subjective wellbeing is collected through self-report and Classical Test Theory lacks detailed information on the scales being considered (Oishi, 2007), psychometric validation using IRT should be the cornerstone of research on subjective measures.The large sample size in this study enabled an IRT analyses and it provided important information on the psychometric properties of the survey.
The validation of the survey produced mixed findings.The factorial structure was mostly consistent with the theoretical development of the questionnaire, particularly for the 'People in Your Life', 'Being a Care Leaver' and 'Feelings' domains.The greatest divergence was for the 'Living Independently' domain, which appeared to split those questions that asked about whether care leavers felt settled and safe in their accommodation and questions to do with economic prosperity.Several questions did not load onto any factors, showed cross-loadings, or loaded on a theoretically inconsistent factor.For example, whether the young person had a pet loaded onto three factors.The loadings might suggest that some young people thought of their pet as a member of their family or suggest that their pets provided emotional support or that pets could be afforded.The responses to whether the young person had a pet also showed invariance across the ethnicity variable.It is a reminder that the cultural context needs to be considered when reporting any survey results.
Positively, the reliability of the total scale was good in terms of both the classical and Rasch model-based statistics.The alpha and the omega suggested a good level of homogeneity across the full questionnaire.The domain-level scales tended to have lower reliability, and this was particularly pronounced for the 'Being a Care Leaver' domain, which had low reliability.When reporting results to individual LAs, the questions that make up that domain are not summed or used as a subscale.Instead, they are used to describe the frequency of responses to key areas that were identified by care leavers as important.They also enabled a comparison between the percentage of positive responses to individual questions, the average response from all the local authorities who had taken part and responses from children in care to similar questions.For example, the psychometric analysis found that the 'number of workers in the previous year' had the worst overall fit.Data from the care leaver survey showed that 60 % of care leavers had the same leaving care worker during the year with little variation in the number of workers.However, only 34 % of children in care aged 11-18 years retained the same social worker in the year with many having three or more workers.The question provided important information on the stability of workers within the LA and whether there were opportunities to develop trusting relationships.National data provides information on the turnover of the social work workforce at the LA level but does not provide the information at the child level (i.e.how much change individual children experience).Similarly, the 'Support' question that asked about who was providing support showed misfit across multiple criteria so it could be argued should be removed or replaced with a question that asked if care leavers were satisfied with the total amount of support.However, LAs were interested in knowing who was providing support e.g., birth family, previous foster carers, or different types of services.Validation statistics (which might suggest the removal of a question) and useful information for practice did not always concur.
The survey total and domain-level scores correlated with the ONS 4 in the expected directions, and this was most positive for the survey total and the 'Feelings' domain.Unsurprising, given that the questions in the 'Feelings' domain are the most alike conventional assessments of subjective wellbeing.The 'Being a Care Leaver' domain had the lowest level of convergent and divergent validity suggesting it is the most different/ unique domain compared to the conventional ONS 4 assessment, and its poorer psychometric properties would also confound these relationships.
The Differential Item Functioning (DIF) analysis indicated that there were differences between males and females for the questions on happiness with appearance and whether they were parents.These sex differences have been highlighted in previous research.(e.g., Mental Health Foundation, 2019; The Children's Society, 2022).Surprisingly, body image is not an area that has received much attention in the social care literature or practice, especially as there is an increased risk of negative body image following maltreatment or trauma.It is an area needing further research.Poorer body image is associated with negative health behaviours (e.g., binge drinking), avoidance of healthcare services and lack of exercise (Bödicker et al., 2022).It should therefore be part of the conversations that workers have with young people.
The psychometric validation has proved very helpful to the Bright Spots Programme.It has enabled the team to step back and think about how the survey could be improved.The response options (always/most of the time, sometimes, and hardly ever/never) to the questions in the 'Being a Care leaver 'domain have been separated into five-point Likert scales, as the initial three points were found to be very easy to agree with/ endorse.The question that asked whether the care leaver was pregnant or a parent was also moved from the domain to the demographics.
The survey findings highlighted that care leavers who reported a disability or long-term health condition had lower scores on most questions compared with other care leavers.Their experience appeared distinctive.It was surprising that those reporting a disability identified the largest number of different types of people providing support but also felt the least emotionally supported.Perhaps those tasked with supporting care leavers avoided asking about mental health or mood.Frontline staff, commissioners, and service managers should understand how this group of care leavers are doing and provide training for leaving care workers.
However, those with a disability/health condition are invisible in the system.Data on the numbers with a disability or the type of disability are not published at a national level on children in care or on care leavers (Hill et al., 2015).Neither are data collected systematically at a local level.Therefore, service commissioners and national decisionmakers should actively identify care leavers who have a disability/ long-term health condition and work with them to understand more about their experiences and how services can be more responsive to their needs.
The difference in wellbeing between those in care and care leavers was also striking.The 'cliff edge' of care and the large drop in positive responses from care leavers suggest that the gains made in care are being squandered by insufficient support.Research has long shown the need for the transition to independence to be gradual and go at the young person's pace and not be done in a hurry (Baker, 2017).The rationale for some services ending at 18 years of age should be challenged, as there is a need for ongoing support.
The LA variation in their care leaver's wellbeing highlights that some LAs were able to provide a good experience but that others continued to experience the territorial injustice identified long ago by Stein (2012).The Bright Spots survey enabled LAs to hear from a larger proportion of their care leavers and the validation statistics presented here, give confidence in the results.The psychometrics supported the use of a total subjective wellbeing scale, but that approach has not been adopted by the survey team.LAs find the results more usefully presented in the domains enabling them to work on specific practice and policy improvements.Other researchers (e.g., Krueger & Stone, 2014;Toussaint et al., 2022) have also argued, that until more progress is made towards developing a credible index of subjective wellbeing the key wellbeing components should be kept distinctive.Findings show that the survey is a valid tool in assessing the subjective wellbeing of care leavers and provides information that enables LAs and social workers to identify areas where care leavers are doing well but also areas where practice and policy need to change.

Limitations
The results are limited by only 31 care leavers co-producing the survey and a minority (13 %) of English local authorities distributing the survey to their care leavers.Females were overrepresented in the respondents, as they are in many surveys.The co-production did produce a useful survey and there have been no negative comments from those completing on the question wording.
There is still much to learn about how subjective wellbeing should be measured.More evidence is needed on the validity of comparing results from vulnerable groups with those from their peers in the general population or comparing results of adolescents with young adults.There is also much to learn about how respondents understand questions and therefore how they respond.For example, do respondents who have had difficult abusive childhoods understand the question, 'Whether the things they do in life have meaning' differently from their peers in the general population?There is also a gap in research evidence in knowing how respondents make judgements on where they sit on scales or fivepoint options, especially when the word 'somewhat' is used in a Likert-type scale.How is 'somewhat' understood?
There are also concerns about validity when a respondent cannot complete a survey without support.In our work, we encouraged LAs to reach all their care leavers and identify a 'trusted adult' who could sit with the care leaver if they needed support and help in completing the questions e.g., if the young person was disabled, could not read English or had other communication difficulties.Recent research (Kooijmans et al., 2022) suggests that in those circumstances respondents tend to answer questions more positively.There is a tension between validity and inclusion but also a challenge that we need to rethink our survey methods.Rather than trying to 'fit' respondents into one method of collecting wellbeing data, methods that enable more respondents to participate fully should be developed.

CRediT authorship contribution statement
(a) Overall, how satisfied are you with your life nowadays?(b) Overall, to what extent do you feel the things you do in your life are worthwhile?(c) How happy did you feel yesterday?(d) How anxious did you feel yesterday?

Fig. 2 .
Fig. 2. Expected value curve for the number of workers the care leaver had during the year.

Table 1
Representativeness of the sample.

Table 2
The ONS 4: very high and low ratings responses from care leavers, the general population, and children in care.

Table 3
Standardized factor loadings from Exploratory Factor Analysis with oblimin rotation.
Note.Loadings < 0.3 have been suppressed in the table.Parent loading is negative because not being a parent at such a young age is known to be a protective factor.1 Questions have been shortened and are not the full questions that appear in the survey.

Table 5
Model comparison statistics for the DIF models compared to the no-DIF model.