1.1 Social connectedness in adolescent students

In adolescence, a critical period for hormonal changes potentially associated with symptoms of psychological distress, friendships develop involving emotional closeness which fulfils adolescents’ need for intimacy, a sense of belonging, and self-validation. It is the period when adolescents seek more independence from their families as they spend more time and develop intimate relationships with friends [8, 9]. Given that for adolescent students most of the friendships develop in school, feeling that one is well-integrated into friendship networks within the school environment and having a strong sense of belonging in school, is expected to have a beneficial effect on an adolescent student’s mental health.

It has been reported that students’ relationships with their parents, school peers, and teachers are linked to their psychological health and students with lower quantity and quality of peer relationships are at higher risk of developing psychological symptoms [10, 11]. Middle school students face significant challenges which can affect their mental health. With peer influence particularly pronounced during this period and the impact of family progressively fading, the school social environment plays an important role in regulating students’ mental health [12].

Several empirical studies have investigated the relationship between social connectedness and health in general, while empirical studies on the relationship between mental health and social connectedness in adolescent students, were designed to estimate effect sizes and were not narrowly focused with respect to the conceptualization of mental health, are few. For example, Diendorfer et al. [13] reviewed 33 studies, mostly in the US, aiming to investigate differences in social connectedness of children and early adolescents with mental disorders, versus those that develop neurotypically. About half of the studies were on autism spectrum disorder (ASD), several on attention deficit hyperactivity disorder (ADHD), and three on social anxiety disorder (SAD). Some other studies on adolescents focused on specific health outcomes in their association with social networks, such as alcohol consumption and suicide risk.

Studies with a broadly similar focus as this study but differing in methodological approach include Ueno [9], who used US National Longitudinal Study of Adolescent Health data and investigated whether those who are integrated into friendship networks have better mental health, as measured by the number of depressive symptoms. Findings from hierarchical linear models supported the general proposition that adolescents have fewer depressive symptoms when they are integrated into friendship networks at school, by finding that higher levels of integration were associated with fewer depressive symptoms. It was also found that adolescents with a strong sense of belonging in school had fewer depressive symptoms and that sense of belonging had a more direct effect on mental health symptoms than did the number of friends, supporting the argument that integration contributes to mental health by strengthening a sense of belonging. More recently, Santini et al. [14] used a large cross-sectional national sample of high school students in Denmark and multilevel regression analysis to assess associations between social disconnectedness in school and various mental health outcomes (including depression symptoms, anxiety symptoms, stress, and sleep problems). They found that social disconnectedness was positively associated with mental health problems and that this association was stronger with each addition in types of social disconnectedness.

1.2 Bidirectional causal pathways

While the public health literature has clearly established that social connectedness promotes mental and physical health, interest in a reverse causal relationship (deterioration in social connectedness because of mental health problems) is found mostly among clinicians. Studies intending to test bidirectional relationships and demonstrate the direction of causal effects between the two constructs are rare. While some studies on reciprocal relationships exist, evidence on adolescents is scarce.

Lester et al. [15] investigated the temporal association between feeling connected to school and mental health prior to and over the transition period from primary to secondary school. They used path analysis to model relationships between school connectedness, depression, and anxiety. Their findings suggest reciprocal relationships between connectedness and mental health, i.e., increased connectedness to school is associated with decreased depression and anxiety; conversely, increased depression and anxiety is associated with decreased connectedness to school. Ding et al. [16] investigated the relationship between community participation and mental health, using Australian nationally representative panel data. They intended to quantify the relationship between past mental health and future community participation, and vice versa. They found that better past well-being led to significant increases in informal social connectedness and civic engagement but decreases in political participation. Yu et al. [17] used data from the British Household Panel Survey and multilevel cross-lagged structural equation analysis to investigate the reciprocal relationship between social capital and health while controlling for demographic characteristics. Their findings suggest that social participation predicts subsequent change in perceived mental health. They also provided evidence of reverse effects, with both perceived mental and physical health appearing to be the dominant causal factor with respect to the prospective level of social networks. Saeri et al. [18] used a large longitudinal sample from the New Zealand Attitudes and Values survey and cross-lagged panel analysis to assess the bidirectional longitudinal relationship between social connectedness and mental health, controlling for baseline levels of both variables and demographics. Social connectedness was found to be a stronger and more consistent predictor of mental health than mental health was of social connectedness.

1.3 Objectives and contributions of the study

Despite the increased availability of longitudinal data that allow for tests of bidirectionality among constructs, there are at least two weaknesses associated with earlier research on the relationship between social connectedness on health, when the objective is deriving and comparing effect sizes. One relates to attributing causality to empirical findings, due to unobservables/omitted variables and associated biases. As assessed by Umberson and Montez [6]: [past research] “…is vulnerable to misattributed causality…other factors not directly assessed may contribute to social ties and health behaviours in ways that produce observed links between the two, real or not…” The endogeneity problem is well known, but inadequately addressed in most empirical studies. Although potential biases due to endogeneities are often pointed out as a limitation, researchers have little idea of the magnitude or even direction of the bias [19]. It requires using more specialised techniques to deal with complications due to unobservables. The other weakness generally encountered in empirical research on social connectedness and health, relates to failure to account for potentially important covariates, sometimes because they are not available in the dataset being used, or because such variables are not frequently used in sociological research. An example is psychological traits potentially associated with health behaviour as well as social connectedness. Similarly, when the social connectedness construct relates to adolescent students’ connections at school and outside the school, it would be useful to control for adolescents’ quality of connections with parents. Furthermore, when the research interest is in mental health, it is important to condition for overall health and related information. Finally, in research on adolescents, measures for adolescents’ various behaviours and habits such as the intensity of interest/use of information and communication technology (ICT), would be relevant as mediators of both psychological health and social relationships. For example, already lonely people may resort to using computers, which could result in fewer real-life contacts and relationships. Also, high-intensity computer use could have a negative impact on health, with symptoms such as headaches, body pains, sleep disorders, tiredness, etc., which in turn could trigger depressive feelings. However, computer use could also improve mental health through access to social support during times of depression [20].

Of the empirical approaches used which aim to uncover a causal relationship, especially while using observational data, some (e.g., multivariate regression, multilevel modelling, and propensity score matching), do not account for the possibility that one or more important confounders, observed or unobserved, could be another common cause of the outcome and the covariate of interest. There are, however, approaches which allow estimating a quasi-causal effect from data that arise from non-experimental research designs [21]. These include quasi-experimental strategies such as the potential outcomes/counterfactual theory of causality [22], the use of natural experiments, and instrumental variables (IV) approaches, which allow considerable control over omitted variables. A recent study [23] followed such an empirical strategy using cross-country European Social Survey data on adults and subjective general health as an outcome. They estimated an instrumental variable model and found that changes in social isolation influence subjective general health.

The specific objectives of this study are to model adolescent students’ perceptions of social connectedness and mental health as potentially endogenous covariates (by endogenous, I refer to a covariate appearing in a model which is correlated with unmeasured confounders which also affects the outcome), when estimating bidirectional effects, to account for potential biases due to endogeneity. For example, with respect to mental health, relevant unobserved confounders (the effects of person-to-person variation that have not been controlled for) include genetic and environmental factors. As reported in the relevant literature, the heritability estimates for anxiety and depression–among the most common emotional and behavioural problems in childhood and adolescence ‐ are 30–40%. Additional unobserved factors relate to individual-specific environmental influences, which sometimes start in the womb [24]. Similarly, with respect to social connectedness, unobservable selection factors may relate to the interaction between genetics (genetic homophily–the genotype of one individual student being predictive of the genotype of his/her friends) and the school environment. The school environment can influence the extent of friendship formations through the mechanism of homophilous relationships [25, 26]. Finally, an additional potential source of endogeneity is measurement/misreporting error associated with the covariate of interest. In the presence of classical (non-systematic) measurement error, effect estimates will generally be biased downwards, while with systematic measurement error, the direction of the bias cannot be easily predicted.

To address various possible potential sources of endogeneity of covariates of interest, an instrumental variables (IV) modelling approach was followed, supplemented with a recently developed alternative approach to deriving effect estimates in the presence of endogeneity and testing the exclusion restrictions of individual instruments. I also exploit the rich information available in the dataset, which allows for conditioning for a wide array of information on adolescent students’ personal circumstances, including self-assessed health, self-reported personality-related attributes, quality of relationships with parents; and school characteristics. Finally, models are estimated separately by gender, since the role of unobservables and associated biases may differ by gender, resulting in different effects by gender in any direction.

In this study, the social connectedness construct (defined in detail in section 2.2.1) encompasses adolescent students’ sense of belonging in school and satisfaction with their friendships in general. School connectedness (or sense of belonging in school) has been defined as ‘the extent to which students feel personally accepted, respected, included and supported by others in their environment’ [27].

The following research questions were investigated:

RQ1: Is perceived social connectedness (sense of belonging in school/satisfaction with friends) an endogenous covariate when estimating effects on mental health? Similarly, is perceived mental health an endogenous covariate when estimating effects in the opposite direction?

RQ2: Can reciprocal effects be established between social connectedness and mental health of adolescent students, and what is the direction of the dominant effect?

RQ3: Are there gender differences in findings associated with research questions 1 and 2?

2.1 Data

I used the 2018 Program for International Student Assessment (PISA-2018) survey data on 15-year-old students. The 2018 version of PISA, besides the compulsory student and school questionnaires which contains information on students themselves and their family, included questionnaires on different aspects of students’ experiences in school, and students’ attitudes, dispositions, and beliefs. PISA 2018 also included an optional questionnaire on students’ well-being, with nine countries participating (Bulgaria, Georgia, Hong Kong, Ireland, Mexico, Panama, Serbia, Spain, and the United Arab Emirates). The well-being questionnaire focused on students’ perceptions of their physical and psychological health, subjective well-being indicators, social relationships, and activities outside of school. The well-being questionnaire assessed important factors for the well-being of adolescent students, both objective (such as health, living environments, and financial situation) and subjective (such as individual personality attributes). Among other well-being indicators of students as individuals (i.e., health, education, and psychological functioning), satisfaction with one’s body image is included in the well-being questionnaire. It also included an optional ICT familiarity questionnaire with questions on different aspects related to digital media and digital devices, internet access, and related questions.

2.2 Estimating causal bidirectional effects: Methodological approach

When using observational data, to meaningfully estimate models involving an outcome variable (y) and a vector of explanatory variables (x), assumes that: (a) vector x is uncorrelated with the error term in the equation to be estimated; (b) there is no simultaneity/reverse causation; and (c) variables in x are measured without error. When we expect violation of one or more of the above requirements ‐ in isolation or in combination with one or more of the covariates in x ‐ covariate endogeneity is a complication. For example, thinking in terms of the relationship between perceptions of social connectedness and mental health, the correlation between perceptions of social connectedness and the error term could arise because we have omitted one or more important covariates from our model which are correlated with perceptions of social connectedness; alternatively, it could be that perceptions of social connectedness and mental health symptoms are to some extent determined by the same unobserved factors. To address the objectives of the investigation, the methodological approach consists of three steps, as outlined below.

Derive a plausible range of degree of endogeneity of the potentially endogenous covariate: Using candidate instrument/s assumed to be relevant and valid (whose relevance and validity are later evaluated), I estimated Extended Regression Models (ERMs) with one covariate assumed to be endogenous. Such models, developed after Heckman’s [31] original work on causal inference models and related work [22], incorporate selection on both observables and unobservables.

The model to be estimated is as follows: (1) (2)

The outcome in each direction of the relationship is determined by the potentially endogenous covariate w₁, other covariates (vector x₁), and error term ε₁. Having identified candidates for instrument/s (x₂), the potentially endogenous covariate, w₁, is determined by a vector of variables that are correlated with w₁, but not correlated with y, except through their effect on w₁ (instrument vector x₂), along with variables in vector x₁, and error term ε₂. Unobservables are represented by the error terms in the four equations.

The error covariance matrix is:

i.e., from this model, an estimate of the correlation in error terms (unobservables) between the two equations (ρ₁₂) is derived, along with a confidence interval.

Test exclusion restrictions at plausible degree of endogeneity, using kinky least-squares (KLS):

Excluded instruments need to be correlated with the endogenous regressor, but uncorrelated with the outcome. The exclusion restrictions of candidate instruments (individually, or in combination) can be tested using the recently developed kinky least-squares (KLS) approach [32, 33]. The main objective of kinky least-squares (KLS) is to provide an instrument-free alternative method of identifying coefficients in the presence of endogenous covariates. KLS yields: (a) statistical inference on the validity of exclusion restrictions regarding candidate external instruments, for a plausible range of endogeneity correlations. I used the point estimate and associated confidence interval of endogeneity correlations derived in step (1), and (b) instrument-free causal effect estimates for the range of values of plausible degree of endogeneity. These estimates come with substantially narrower confidence intervals compared to the IV estimates.

Derive bidirectional effect estimates from IV estimation: Using the proposed instruments, IV estimates of bidirectional effects were derived. When more than one instrument is available, in addition to the KLS-based exclusion restriction tests for individual instruments, conventional overidentification tests of instrument combination are reported (overidentification tests of all instruments).

2.3 Proposed instruments

In the direction from social connectedness to mental health (Model 1), the instrument used is the school-level average response (derived using the country-school identifier in the survey) to the question “I feel like I belong in school”, with responses ranging from strongly disagree to strongly agree. In the direction from mental health to social connectedness (Model 2), the main instrument set is based on responses to questions on how students felt when last attended math or language classes (Bored; Challenged; Nervous/Tense; Motivated/Inspired), with responses ranging from ‘Not at all, to Extremely). In the male regression, a combination of two instruments was used: feeling Nervous/Tense when attending a math class and feeling bored when attending a language class. In the female regression, feeling Nervous/Tense when attending a math class was used as a single instrument.

In model 2, an alternative ‐ school-level ‐ instrument was considered (see section 3.4). For males, the instrument is based on Yes/No responses at the school level to questions on how they felt the day before (i.e., “accomplished something”; “were treated with respect”; “smiled or laughed”; “learn something interesting”; “had enough energy to get things done”; “overall, satisfied with their day”). The instrument is the school average of: “Yesterday, I had enough energy to get things done”. For females, the instrument is based on responses about attitudes towards competition at school (“I enjoy working in situations involving competition with others; “It is important for me to perform better than other people on a task”; “I try harder when I’m in competition with other people”), with responses ranging from strongly disagree to strongly agree. The continuous ordered instrument used is the school mean of: “It is important for me to perform better than other people on a task”. Higher values indicate a more competitive school environment.

Given that the instruments used are ordered, there is a different local average treatment effect (LATE) parameter for every value of the instrument; hence, the linear IV estimator estimates a weighted average of local average treatment effects.

Instrument relevance and validity depend on the following main assumptions:

Relevance: The instrument and the endogenous covariate are sufficiently associated, either because of a causal association between the two or because the two share a common cause. In observational studies, it is assumed that there is an unmeasured causal instrument which is the common cause of the measured proxy instrument and the endogenous covariate [34]. In Model 1, the intuition is that the sense of belonging at the individual student level will be correlated to the sense of belonging at the group/school level. In Model 2, the more intense the feeling of nervousness/tension when attending a math class is expected to be negatively correlated with students’ mental health. With respect to the alternative instruments in Model 2, belonging to a group whose members are on average more satisfied with their daily lives at school, is expected to be positively correlated with the perceived mental health of an individual member. Finally, belonging in a more competitive school environment is expected to be negatively correlated with the perceived mental health of an individual student.

Independence: The instrument must be uncorrelated with the error term, i.e., the instrument does not share common causes with the outcome. Independence cannot be tested, since the error term is, by definition, unobservable.

The exclusion restriction: The instrument is independent of the outcome, after conditioning for additional covariates, i.e., the instrument affects the outcome only through its effect on the endogenous covariate.

An additional assumption needed to point-identify a causal effect is that of Monotonicity. This assumption is usually in the context of binary instruments [35, 36]. Deterministic monotonicity states that for each subject, the level of the treatment that a subject would take is a monotonic increasing function of the level of the instrument. This assumption is unlikely to hold in most cases. A weaker version of monotonicity, that of Stochastic Monotonicity, only requires a monotonic relationship between the IV and the probability of treatment, hence it is easier to satisfy. Small et al. [37], show that under stochastic monotonicity, the IV method identifies a weighted average of treatment effects.

Given the expectation that the instrument validity assumptions do not hold precisely, in section 3.4 (Further analysis), I implement the procedure suggested by Nevo and Rosen [38] which relaxes the key IV correlation assumption, by replacing the assumption of zero-correlation between the instrument and the unobserved error term with an assumption on the “sign” of the correlation, thus allowing for the construction of IV bounds under weaker-than-traditional assumptions.

2.4 Summary statistics

Table 1 contains the summary statistics of outcome variables and covariates by gender. Female participants reported significantly worse mental health complaints than male participants. From the related questionnaire items, the frequency of reported symptoms across all related questions for female participants is about 1.7 times higher than for male participants. On the other hand, perceptions of social connectedness do not differ significantly between male and female participants.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Summary statistics by gender.

https://doi.org/10.1371/journal.pone.0294591.t001

Males assess their health as “excellent” at substantially higher rates than females (48% vs. 33%), while a higher proportion of females assess their health as “fair” (16% vs. 9%). Females reported significantly lower body image satisfaction than males. Females reported a higher fear of failure, while males reported substantially less emotional support from, and worse social connections with parents, compared to females. Finally, based on their experiences in school, males reported a worse disciplinary climate in school (such as noise and disorder in the classroom).

2.5 Instrument testing

When more than one instrument was available, instrument relevance and exclusion restrictions were assessed using conventional tests from IV estimation. In addition, I used the kinky least-squares (KLS) approach, which yields statistical inference on the validity of exclusion restrictions regarding candidate external instrument/s (single, or as a set) when a plausible range of endogeneity correlations is available, whereas these restrictions were always considered non-testable. In the IV framework, if the model is ‘just identified’ (i.e., one instrument for one endogenous regressor) the exclusion restriction cannot be tested; IV estimators assume that the direct effect of the instrument on the outcome is 0. Even in ‘overidentified’ models (i.e., at least two instruments for one endogenous regressor), tests of overidentifying restrictions rely on the (untested) assumption that at least as many instruments are validly excluded from the model as endogenous regressors [39]. The KLS approach (implemented using the kinkyreg module in Stata) supplements standard IV inference by allowing assessment of the validity of previously untestable “just-identifying” exclusion restrictions [33]. In other words, if instruments are available, this approach allows for sensitivity checks for IV-based estimation. In this sense, it supplements IV estimation.

Instrument testing using KLS requires prior knowledge of a plausible range of degree of endogeneity (endogeneity correlations) of the endogenous covariate; to derive estimates of plausible endogeneity correlations, I estimated Extended Regression Models (ERMs) with one endogenous covariate (using the eregress module in Stata), and the same set of instruments as in the IV regressions. Given a set of instruments, ERMs produce coefficient estimates which closely mimic those from the IV estimation; in addition, they produce estimates and confidence intervals of endogeneity correlations (correlations of error terms between the outcome equation and the equation determining the potentially endogenous regressor), which served the purpose of a plausible range of endogeneity correlations when testing exclusion restrictions using KLS.

3.1 Model 1: From social connectedness to mental health

Table 2 contains the OLS and IV results by gender, with the social connectedness composite as a potentially endogenous covariate. The top panel contains the coefficient estimates, while the middle panel contains the first-stage results (coefficient estimates of the model determining social connectedness). The bottom panel gives the F-statistic for excluded instruments and test of endogeneity of social connectedness. With a single instrument, the exclusion restrictions were tested within the KLS framework, for a plausible range of degree of endogeneity of the social connectedness composite derived from estimating ERMs.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. From social connectedness to mental health: Effect estimates by gender.

https://doi.org/10.1371/journal.pone.0294591.t002

The estimates of the effect of better social connections on mental health from multiple linear (OLS) regressions are small; one SD increase in the social connectedness composite is associated with about 0.1 SD increase in the mental health composite. The small difference in estimates by gender is not statistically significant. These effect estimates are of similar magnitude to those from other studies which rely on longitudinal data but do not consider endogeneity biases [18].

Columns 2 and 3 in Table 2 report estimates adjusted for endogeneity of social connectedness perceptions from two approaches, one instrument based (IV) and the other instrument-free (KLS). Given that KLS estimates are unweighted, they come with much smaller standard errors compared to the IV and the OLS estimates, but since they are unweighted, they are inconsistent. The point estimates and 95% confidence intervals of endogeneity correlations are -0.067 [95% CI: -0.13, 0.01] for males and -0.131 [95% CI: -0.20, -0.04] for females.

Figs 1 and 2 depict the KLS estimates and exclusion restriction tests for males, while Figs 3 and 4 are the corresponding figures for females. Figs 2 and 4 depict the p-values for Wald tests at various values of the postulated endogeneity correlations. For males, the hypothesis that the instrument is validly excluded from the model is accepted for the [-0.11, 0] range of endogeneity correlations, while for females the corresponding range of acceptance range is [-0.176, -0.05]. At the point estimates of -0.067 and -0.131 for males and females respectively, the hypothesis is accepted at high p-values.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Model 1 KLS estimates: Males.

https://doi.org/10.1371/journal.pone.0294591.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Model 1 exclusion restriction test: Males Endogeneity correlation point estimate: -0.066.

https://doi.org/10.1371/journal.pone.0294591.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Model 1 KLS estimates: Females.

https://doi.org/10.1371/journal.pone.0294591.g003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Model 1 exclusion restriction test: Females Endogeneity correlation point estimate: -0.13.

https://doi.org/10.1371/journal.pone.0294591.g004

The IV (2SLS) estimates of the effect of social connectedness on mental health are somewhat larger than the OLS estimates, but as is usually the case, less precisely estimated (β = 0.20; 95% CI = [-0.038, 0.43] for males and β = 0.28; 85% CI = [0.066, 0.50] for females). These estimates are essentially identical to the KLS estimates (0.21 for males and 0.28 for females). While the IV estimates are somewhat larger for females compared to males, the difference is not statistically significant.

Endogeneity tests accept the null hypothesis of exogeneity of the social connectedness composite in the male sample. In the female sample, there is evidence of endogeneity, with an associated p-value of 0.09. Finally, tests of instrument relevance indicate that the instrument is not weak. Using the rule of thumb, the first stage F-statistics, at 57 and 81 respectively for males and females, far exceeds 10.

3.2 Model 2: From mental health to social connectedness

Table 3 reports results in the direction from mental health to social connectedness. IV estimates are based on instruments related to how students felt when they last attended a math/language class. Results from using alternative instruments are discussed in section 3.4 (Further analysis). Multiple linear regression estimates (column 1) are small and similar to those in the opposite direction, with effect sizes ranging from 0.09 to 0.16 for female and male participants respectively. The gender difference in associations is statistically significant at the 5% level.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. From mental health to social connectedness: Effect estimates by gender.

https://doi.org/10.1371/journal.pone.0294591.t003

However, IV estimates (column 2) are about 5 times larger than the OLS estimates (β = 0.58; 95% CI = [0.28, 0.88] for males β = 0.43; 95% CI = [0.19, 0.66] for females) and the gender difference in effect size is not statistically significant. Effect estimates from KLS present a similar picture (0.64 for males and 0.49 for females). These large effect sizes reflect the strong evidence of endogeneity of mental health perceptions. Point estimates and confidence intervals of error term correlations between the outcome and the equation modelling mental health perceptions are strongly negative (-0.37 [95% CI: -0.55, -0.15] for males and -0.32 [95% CI: -0.39, -0.25] for females). The negative error term correlations indicate that unobservables which tend to improve mental health perceptions occur with unobservables which tend to worsen social connectedness perceptions.

In the male sample, the joint null hypothesis that the instrument set is uncorrelated with the error term and that the excluded instruments are correctly excluded from the estimated equation is accepted at high p-values. Further evidence is given from KLS-based estimates and exclusion restrictions tests of each individual instrument (Figs 5–8). Using 95% confidence intervals of the degree of endogeneity compatible with the valid exclusion of the instrument, these intervals are [-0.43, -0.31] and [-0.53, -0.22] for the two instruments in the male regression. In the female regression, with one instrument, the 95% confidence interval for the degree of endogeneity compatible with the valid exclusion of the instrument is [-0.34, -0.21]. Finally, the first stage F-values of excluded instruments in the IV regressions (47 and 90 for males and females, respectively) indicate that the instrument set is not weak.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Model 2 KLS estimates: Males.

https://doi.org/10.1371/journal.pone.0294591.g005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Model 2 exclusion restriction test: Males Endogeneity correlation point estimate: -0.37.

https://doi.org/10.1371/journal.pone.0294591.g006

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Model 2 KLS estimates: Females.

https://doi.org/10.1371/journal.pone.0294591.g007

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Model 2 exclusion restriction test: Females Endogeneity correlation point estimate: -0.32.

https://doi.org/10.1371/journal.pone.0294591.g008

Overall, the IV and KLS estimates from the two models tell a consistent story. In the direction from social connectedness to mental health, evidence of endogeneity of social connectedness perceptions is weak, especially in the male sample; hence, after accounting for endogeneity biases the effect of social connectedness on mental health is modest and of similar magnitude for male and female participants. However, in the direction from mental health to social connectedness, accounting for biases associated with the endogeneity of mental health perceptions is important. Comparing effect sizes, the dominant effect is from mental health to social connectedness.

3.3 Other covariates

Looking at significant associations between covariates and the outcome in the two models, in Model 1 the larger associations are found between better overall health and better mental health. The estimated effect sizes are large, with more than one SD better mental health when the comparison is between assessing health as excellent vs. poor. On the other hand, in the opposite direction, better overall health is only weakly associated with social connectedness perceptions. Looking at other covariates, among student demographic and family characteristics, being of immigrant background is negatively associated with social connectedness (and more so for female participants). Reporting more interest in ICT is negatively associated with mental health perceptions; however, the opposite is the case for its association with social connectedness. Among psychological constructs, reporting a stronger fear of failure is negatively associated with mental health perceptions. On the other hand, reporting higher resilience associated with better social connectedness, but no association between resilience and mental health was found. Finally, among school-level variables, the association between male students attending a single-sex school and both mental health and social connectedness perceptions is negative, but this is not the case for females for either of the two outcomes.

3.4 Further analysis

3.4.1 Assuming imperfect instruments.

I derived IV bounds on estimates when assumptions underlying instrument validity do not hold precisely, i.e., relaxing the zero-covariance assumption between the instrument and the error term, using the Nevo and Rosen [38] approach. Given an endogenous covariate of interest, x, and an instrument, z, instead of assuming that ρ_zε = 0, it is assumed that the instrument has (weakly) the same direction of correlation with the omitted error term as the endogenous covariate (i.e., ρ_xερ_{zε ≥} 0). If the instrument is less endogenous than the endogenous covariate of interest (i.e., ρ_xε ≥ ρ_zε), then this amounts to essentially considering an “imperfect” instrumental variable.

Bounds from IV estimation assuming imperfect instruments (derived using Stata’s imperfectiv module by Clarke and Matta [40] are given in Tables 4 and 5. As expected, the uncertainty associated with the bounds is larger. In Model 1, the lower bound CI includes the OLS estimates. This is not surprising, given that the evidence of endogeneity of the social connectedness composite is weak. In Model 2, the lower bound CI excludes the OLS estimates for both male and female participants; however, the overall 95% CI of estimates are wider, i.e., [0.17, 0.82] for males (compared to [0.28, 0.88] in the main results), and [0.11, 0.62] for females (compared to [0.19, 0.66]), due to wider lower bounds. Overall, the implications of findings regarding the relative size of effect estimates between Model 1 and Model 2 strongly support that the dominant effect is in the direction from mental health to social connectedness.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Nevo and Rosen (2008)’s Imperfect IV bounds–Model 1.

https://doi.org/10.1371/journal.pone.0294591.t004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Nevo and Rosen (2008)’s Imperfect IV bounds–Model 2.

https://doi.org/10.1371/journal.pone.0294591.t005

3.4.2 Sensitivity to instrument choice in Model 2.

The alternative instruments used in IV estimation in Model 2 are school-level instruments. For males, it is the school mean for agreeing with the statement: ‘Yesterday [a typical day] I had enough energy’; for females, it is the school mean response (strongly disagree, to strongly agree) to the statement: ‘it is important for me to perform better than other people.’ As expected, the first is positively correlated with individual students’ mental health (i.e., less mental health-related complaints), while the second (i.e., more competitive school environment), is negatively correlated with individual students’ mental health. Figs 9–12 contain the information from testing the exclusion restrictions (95% CI of the degree of endogeneity of the mental health composite compatible with valid exclusion) for the alternative instrument. Figs 13–16 contain the corresponding information when the alternative instrument is combined with the main instrument/s (i.e., those behind the results in Table 3).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Model 2 KLS estimates with alternative instrument: Males.

https://doi.org/10.1371/journal.pone.0294591.g009

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Model 2 exclusion restriction test with alternative instrument: Males.

Endogeneity correlation point estimate: -0.36.

https://doi.org/10.1371/journal.pone.0294591.g010

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Model 2 KLS estimates with alternative instrument: Females.

https://doi.org/10.1371/journal.pone.0294591.g011

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 12. Model 2 exclusion restriction test with alternative instrument: Females.

Endogeneity correlation point estimate: -0.39.

https://doi.org/10.1371/journal.pone.0294591.g012

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 13. Model 2 KLS estimates with combination of instruments: Males.

https://doi.org/10.1371/journal.pone.0294591.g013

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 14. Model 2 exclusion restriction test with combination of instruments: Males.

Endogeneity correlation point estimate: -0.36.

https://doi.org/10.1371/journal.pone.0294591.g014

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 15. Model 2 KLS estimates with combination of instruments: Females.

https://doi.org/10.1371/journal.pone.0294591.g015

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 16. Model 2 exclusion restriction test with combination of instruments: Females.

Endogeneity correlation point estimate: -0.44.

https://doi.org/10.1371/journal.pone.0294591.g016

Table 6 contains the IV estimates using the alternative instrument (column 1) and the combination of the main and the alternative instrument (column 2). In column 1, the point estimates of the effect of perceived mental health on perceived social connectedness (β = 0.54 for males and 0.48 for females), are essentially identical to those in Table 3. However, they are substantially less precise (95% CI: [0.05, 1.03] for males and [-0.08, 1.04] for females). In column 2, the combination of three instruments for males and two instruments for females results in more precise point estimates, which nevertheless present the same picture (β = 57; 95% CI = [0.38, 0.76] for males β = 0.53; 95% CI = [0.30, 0.75] for females). Based on the overidentification tests of all instruments (J-test), the hypothesis that the set of additional instruments is exogenous is accepted at high p-values for both males and females.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. From mental health to social integration: Alternative instruments.

https://doi.org/10.1371/journal.pone.0294591.t006

3.4.3 Sensitivity of results to model specification.

Some of the covariates in the models other than the covariates of interest, are based on students’ perceptions about themselves, hence potentially endogenous. I re-estimated the models with other potentially endogenous covariates omitted, keeping only demographic, individual student, and school characteristics. From the estimation results (reported in Tables 7 and 8), the overall picture did not change (large effects from mental health to social connections, modest effect estimates from social connections to mental health). However, effect estimates are 15–20% higher than when controlling for the full set of observable covariates, given that the coefficient estimates of interest now partially reflect the effect of omitted characteristics on outcome. Furthermore, the validity of exclusion restrictions is accepted at lower p-values compared to the model with the full set of controls. These findings seem to support suggestions that the assumptions related to identification may be more plausible after conditioning on covariates and that instruments are sometimes valid only after conditioning for certain covariates [41, 42].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. From social integration to mental health: IV estimates omitting other potential endogenous regressors.

https://doi.org/10.1371/journal.pone.0294591.t007

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. From mental health to social integration: IV estimates omitting other potential endogenous regressors.

https://doi.org/10.1371/journal.pone.0294591.t008

4.1 Implications and policy relevance

The clear implication of the main finding is that the predominance of findings supporting the dominance of the effect in the direction from social connectedness to mental health may be due to failure to account for the influence of unobservables. The intuition for the importance of adolescent mental health in maintaining social relationships is that adolescent students with mental health symptoms can progressively become socially isolated because of negative experiences when interacting with others, as well as negative reactions of others. Mental health can also affect how social situations are perceived which can, in turn, affect one’s behaviour. Given the predominance of earlier findings of a dominant effect in the direction from social connectedness to mental health, policies have been increasingly emphasizing the importance of social relationships for mental health, including adolescent mental health. However, in medical and psychiatric settings, patients’ social functioning is rarely the point of intervention; instead, it is often considered an indicator of impaired mental health [43, 18].

The policy relevance of the findings in this study is that adolescent mental health should be the primary focus of interventions, i.e., identifying and treating mental health symptoms as a primary intervention and as a precursor to improving the social connectedness of adolescents. Schwartz and Litwin [44] also reported dominance of the direction from mental health to social connections, but in older adults, refer to the possibility of a “vicious cycle” in which those in worse mental state may end up with a less supportive social environment, which in turn may result in a deterioration of mental health. This is also relevant to adolescents, given that adolescence and the onset of puberty is a critical period for neurological, social, and cognitive development, potentially associated with symptoms of psychological nature. Onset of mental health symptoms may result in social withdrawal and further mental health impairment.

4.2 Strengths and limitations

I outlined and implemented a quasi-experimental approach to derive effect estimates when the covariate of interest is potentially endogenous. This is often the case with covariates derived from survey participants’ perceptions or when using psychological constructs. As was the case of this study, biases associated with the endogeneity of covariates of interest can be important when looking for answers to research questions.

This study has limitations. The data sample used for analysis comprises nine countries, i.e., the countries that participated in the wellbeing module of PISA 2018. Hence, the findings may not be generalizable; had the estimation sample been larger, findings may have been somewhat different. Finally, it would be preferable to implement the quasi-experimental approach used in this study with longitudinal data.