Beyond factor analysis : Insights into the dimensionality of the Fortitude Questionnaire through bifactor statistical analysis

Adversity is part of the human experience. For some people, the experience of adverse life events is associated with negative psychological outcomes, including depression, anxiety and post-traumatic stress disorder. For others, adversity leads to growth, adaptative functioning and effective coping. To understand this heterogeneity in responses to life stressors, Pretorius (1998) proposed the construct of fortitude, which is defined as the psychological strength to manage stress and stay well. Fortitude arises from three inter-related positive or fortigenic appraisals of self, family and external sources of support. Self-appraisals include both a global positive evaluation of oneself as well as more specific positive appraisals of one’s competence and capacities to manage stressors. Family appraisals entail an evaluation of both the general family environment and family unit as cohesive, facilitative of emotional expression and responsive and accessible in times of stress. Support appraisals include an evaluation of the availability, accessibility and value of support from others (e.g. friends). From this theoretical perspective, Pretorius (1998) developed the Fortitude Questionnaire (FORQ), which measures the psychological strength associated with managing adversity. Since its development, the FORQ has been extensively used in South Africa (e.g., Geldenhuys & Van Schalkwyk, 2019; Padmanabhanunni, 2020), as well as in several other countries, such as Nigeria (Adejuwon, Aderogba, & Adekeye, 2015), Canada (Beattie, Stewart, & Walker, 2016), Indonesia (Yuwanto & Atmadji, 2017) and the United Arab Emirates (Hameed, Khan, Shahab, Hameed, & Qadeer, 2016).


Introduction
Adversity is part of the human experience. For some people, the experience of adverse life events is associated with negative psychological outcomes, including depression, anxiety and post-traumatic stress disorder. For others, adversity leads to growth, adaptative functioning and effective coping. To understand this heterogeneity in responses to life stressors, Pretorius (1998) proposed the construct of fortitude, which is defined as the psychological strength to manage stress and stay well. Fortitude arises from three inter-related positive or fortigenic appraisals of self, family and external sources of support. Self-appraisals include both a global positive evaluation of oneself as well as more specific positive appraisals of one's competence and capacities to manage stressors. Family appraisals entail an evaluation of both the general family environment and family unit as cohesive, facilitative of emotional expression and responsive and accessible in times of stress. Support appraisals include an evaluation of the availability, accessibility and value of support from others (e.g. friends). From this theoretical perspective, Pretorius (1998) developed the Fortitude Questionnaire (FORQ), which measures the psychological strength associated with managing adversity. Since its development, the FORQ has been extensively used in South Africa (e.g., Geldenhuys & Van Schalkwyk, 2019;Padmanabhanunni, 2020), as well as in several other countries, such as Nigeria (Adejuwon, Aderogba, & Adekeye, 2015), Canada (Beattie, Stewart, & Walker, 2016), Indonesia (Yuwanto & Atmadji, 2017) and the United Arab Emirates (Hameed, Khan, Shahab, Hameed, & Qadeer, 2016).
The FORQ has been applied to different populations, such as healthcare workers (Adejuwon et al., 2015), university students (Beattie et al., 2016), adolescents exposed to traumatic events (Pretorius, Padmanabhanunni, & Campbell, 2016), nurses caring for patients with Alzheimer's disease (Heyns, Venter, Esterhuyse, Bam, & Odendaal, 2003) and lay trauma counsellors This study applied confirmatory factor analyses to explore the factor structure of the Fortitude Questionnaire (FORQ) in three samples: adolescents, students and lay counsellors. For the student and lay counsellor samples, the analysis demonstrated that a bifactor and a threefactor second-order model of the FORQ was a much better fit than a one-factor model, whilst in the adolescent sample, there was no discernible difference between the three models. Ancillary bifactor analysis was also conducted to examine the dimensionality of the FORQ. The bifactor measures confirmed that the FORQ is not unidimensional, but rather multidimensional for the student and lay counsellor samples. For the adolescent sample, there are some concerns as the general factor accounted for 77% of the variance, whilst the subscales accounted for only 23% of the variance. Furthermore, in the standardised solution for the adolescent sample, only the factor loadings for the total scale were significant. In addition, the model-based estimates of reliability were low for the self-appraisal and support-appraisal subscales in the adolescent sample. This finding indicates that the FORQ was essentially unidimensional in the adolescent sample. These results suggest that for young adult and adult samples, the FORQ may be utilised as a total scale and three subscales, whilst in adolescent samples, caution needs to be applied in using the FORQ subscales with children and adolescent samples. However, further research that replicates this finding in adolescents and other samples is needed before a definite conclusion about the suitability of the FORQ in different age groups can be reached.

Beyond factor analysis: Insights into the dimensionality of the Fortitude Questionnaire through bifactor statistical analysis
Read online: Scan this QR code with your smart phone or mobile device to read online. (Padmanabhanunni, 2020). It has also been used for a variety of purposes, including to investigate the role of fortitude in mental health-seeking behaviour (Beattie et al., 2016) and in psychological outcomes following exposure to secondary trauma (Padmanabhanunni, 2020), to evaluate a resiliency programme for children (De Villiers & Van den Bergh, 2012), to establish levels of wellbeing amongst adolescents in highrisk communities (Geldenhuys & Van Schalkwyk, 2019) and to assess the effect of programmes intended to enhance psychosocial wellbeing (Van Schalkwyk & Wissing, 2013).
The FORQ has generally demonstrated sound internal consistency (Cronbach's alpha) in previous studies, with a few exceptions mostly related to the self-appraisal subscale (Laureano, Grobbelaar, & Nienaber, 2014;Talbot, 2012). Both exploratory (Yuwanto & Atmadji, 2017) and CFA (Wissing, Du Toit, & Michael Temane, 2008) have provided support for the conceptualisation of the FORQ as consisting of a total scale and three subscales.
Despite this conceptualisation of the FORQ, the questionnaire has been used in a variety of ways in terms of structure, namely as a total scale (Geldenhuys & Van Schalkwyk, 2019), as a total scale with subscales (Padmanabhanunni, 2020), as subscales only (Talbot, 2012) and as selected subscales (Peters, 2005). However, in the original conceptualisation (Pretorius, 1998), fortitude was conceived as arising from the interaction of three domains: self-appraisals, familyappraisals and support-appraisals. It is therefore questionable whether the use of only subscales or selected subscales reflects the original conceptualisation of fortitude.
With respect to scales that are presumed to consist of a total scale and several subscales, it is important to examine whether the subscales account for a sufficient amount of the variance amongst the items to be regarded as independent scores. If the subscale scores have very little specific reliable variance with most of the variance being the variance that is shared with other subscales, they cannot be regarded as independent scores. For example, exploratory factor analysis (EFA) could result in a three-factor solution, and confirmatory factor analysis (CFA) could confirm that such a three-factor model best fits the data. However, the existence of such factors does not address the 'essential' unidimensionality or multidimensionality of the instrument (Raykov & Pohl, 2013). If a scale is multidimensional, it means that (1) a general factor accounts for some of the variance between items and (2) beyond the variance accounted for by the general factor, sufficient variance remains that is accounted for by the subscales. In an essentially unidimensional scale, a general latent variable account for the majority of variance, with only a small proportion of variance accounted for by the subscales (Rodriguez, Reise, & Haviland, 2016a). In this instance, where the general factor accounts for almost all of the variance, the subscale scores should not be interpreted as independent scores. The importance of underscoring the difference between factor structure and dimensionality was highlighted in a study that applied bifactor statistical indices to 50 published studies of different questionnaires (Rodriguez, Reise, & Haviland, 2016b). The researchers concluded that, although all of the measures in the 50 studies had been described as multidimensional, the bifactor indices indicated that the variance in all of these measures was overwhelmingly accounted for by a single latent variable.
The aim of this study is to provide evidence of the dimensionality of the FORQ in three samples through the use of CFA and bifactor statistical indices. No published work to date has investigated the dimensionality of the FORQ, that is, the extent to which sufficient variance is accounted for by the subscales after isolating the variance attributable to the general scale. This kind of information can support or serve as a caution in the use of the total scale and/or subscales and highlight issues that need to be considered when the scale is used amongst specific population groups.

Method Participants
Sample 1: Padmanabhanunni (2020) investigated the role of fortitude in professional quality of life amongst lay trauma counsellors (N = 146) in the Western Cape Province of South Africa. The study employed a cross-sectional survey design and convenience sampling. The participants were lay trauma counsellors who worked for non-governmental organisations providing services in disadvantaged community contexts. The majority of participants were women (76.9%), and the mean age was 44 years.
Sample 2: Pretorius et al. (2016) investigated the role of fortitude in affecting psychological outcomes after exposure to traumatic events amongst adolescents in two low-income communities in the Western Cape Province of South Africa. The participants were adolescents (N = 498) in grades 8-12. The majority of participants were women (51.2%) and Afrikaans speaking (70.2%), and the mean age was 15.1 years.

Sample 3:
The third data set is unpublished to date and focuses on fortitude in relation to psychological well-being. The associated study was conducted amongst undergraduate students (N = 454) at a university in the Western Cape Province of South Africa. The majority of participants were women (71.4%), and the mean age was 25.1 years.

Procedure
Study 1: Participants received information on the nature and aims of the study and a request to participate in the study. Consenting participants were provided with the questionnaires electronically or in person. The response rate was 58%.
Study 2: Self-report measures were administered over a 2-week period. After the researchers determined the language preference of participants, the questionnaires were administered in English or Afrikaans. When the use of Afrikaans was appropriate, the FORQ was translated into Afrikaans and back-translated into English. Participants were provided with information regarding the nature and aims of the study, as well as the content and completion requirements of the questionnaire.
Study 3: Participants received information regarding the nature and aims of the study during their regular classes. Those interested in participating were provided with selfreport measures that were completed anonymously.

Measures
All three studies used the FORQ. The FORQ is a 20-item questionnaire that uses a four-point scale ranging from 'does not apply' to 'applies very strongly'. The scale measures three domains of fortitude: self-appraisals, family-appraisals and support-appraisals. The sum of the three domains represents the individual's level of fortitude. In a validation study, Pretorius (1998) reported coefficient alphas of between 0.74 and 0.82 for the subscales and a coefficient of 0.85 for the full scale. Other South African studies have reported reliability coefficients of between 0.77 and 0.88 (Heyns et al., 2003;Wissing et al., 2008). The FORQ is also correlated with measures of psychological distress and measures of self-appraisal (i.e. self-esteem), social support and the family environment (Pretorius, 1998). In addition to the FORQ, the participants in the three studies completed the measures indicated below.
Study 1: Participants completed two self-report measures: the Life Events Checklist-5 (Weathers et al., 2013) and the Professional Quality of Life Scale (Stamm, 2005).

Data analysis
Confirmatory factor analysis was used to test three conceptualisations of the factor structure in the three samples. In CFA, the items of the scale are regarded as the observed measurements, whilst the hypothesised factors are regarded as the latent variables represented by the items (Bentler, 1995). The three conceptualisations of the factor structure of the FORQ that were examined were a one-factor model (representing a total fortitude score), a three-factor secondorder model and a bifactor model. The bifactor model hypothesised that the FORQ consists of a single general factor with the three subscales as orthogonal factors reflecting the variance amongst clusters of items (Mansolf & Reise, 2017). More specifically, it is the reliable variance that remains after removing the variance attributable to the general factor. In addition, using the Bifactor Indices Calculator (Dueber, 2017), ancillary bifactor measures were calculated to clarify the dimensionality of the FORQ. These measures include (1) explained common variance (ECV), which is the proportion of reliable variance explained by the specific factor; (2) omega, which is a model-based estimate of reliability (omegaS for subscales); and (3) omega hierarchical (omegaH), which indicates the proportion of systematic variance in total scores that can be attributed to individual differences on the general factor. In general, a high omegaH (> 0.80) is an indication that the scale is essentially unidimensional. For subscales, the omegaHS represents the proportion of reliable systematic variance of a subscale score after excluding the variability attributed to the general factor (Rodriguez et al., 2016a).
In CFA, the extent to which the hypothesised model fits the observed data is measured by the chi-square statistic (χ 2 ), which tests the null hypothesis of a perfect fit. Jöreskog, Olsson and Wallentin (2016), however, pointed out that the χ 2 test has too much power in large samples and is very sensitive to violations of distributional assumptions. Kline (2005) suggested that, in addition to the model χ 2 , at a minimum, the following indices should be reported: the root mean square error of approximation (RMSEA: best if close to 0.08 or less), comparative fit index (CFI: best if close to 0.90 or greater) and standardised root mean square residual (SRMR: best if close to 0.08 or less). Additional indices include the goodness-of-fit index (GFI: best if close to 0.95 or greater) and Tucker-Lewis index (TLI: best if close to 0.95 or greater; Byrne, 1994;Hu & Bentler, 1999). Arbuckle (2012) also proposed the inclusion of fit indices such as Akaike's information criterion (AIC), which is used specifically for model comparisons. Lower AIC values are generally associated with a better model fit.
With the exception of the bifactor indices, all analyses were performed using IBM SPSS Amos (version 26; IBM Corp., Armonk, NY, USA).

Ethical consideration
Study 1: Ethical approval for the study was provided by the Humanities and Social Sciences Research Committee of the University of the Western Cape. Assent was obtained from non-governmental organisation (NGO) directors to contact lay counsellors, and each participant completed an informed consent form. The questionnaires contained no identifying information.
Study 2: Ethical approval for the study was provided by the University of the Free State. The parents of the participants provided consent, and all questionnaires were completed anonymously. The nature and aims of the research were described to each class, and confidentiality was assured.

Results
The three models that were tested with CFA in the three samples are presented in Figures 1 and 2.
The one-factor model presumes that a single factor (fortitude) best explains the variance amongst the items, whilst the three-factor second-order model presumes that a secondorder factor (total scale) best accounts for the variance amongst the first-order factors (subscales). The bifactor model, in contrast, presumes that a single general factor (fortitude) explains some of the variance, whilst three specific factors (subscales) account for the remainder of the variance. The fit indices for the three models in the three samples are reported in Table 1.
As detailed in Table 1, there was almost no difference in the fit indices of the three models in the adolescent sample. Whilst the model comparison index (AIC) was lower for the bifactor model (456.54 < 476.45 and 481.13), indicating a marginally better fit than the one-factor and three-factor second-order models, the other indices, including the RMSEA (0.05) and SRMR (0.06), indicated that all three models fit the data to an acceptable degree. In the student and lay counsellor samples, however, the bifactor model demonstrated a better fit than the one-factor model and marginally better than the three-factor second-order model. In the adolescent sample, all the loadings on the general factor (fortitude) were significant. The loadings on the specific factors (subscales), however, were all non-significant, with the exception of item 1 of the self-appraisal subscale. In the student sample, the loadings for the general factor as well as the specific factors were all significant. The same applied to the lay counsellor sample, except for item 6 of the selfappraisal subscale and item 2 of the support-appraisal subscale. Despite the evidence provided by the CFA in relation to the bifactor structure of the FORQ, the CFA did not address the dimensionality of the questionnaire. More specifically, the CFA did not clarify the relative proportion of variance accounted for by the total scale and the subscales. For this reason, some authors have called for the use of bifactor indices to examine dimensionality (Rodriguez et al., 2016a). These indices for the FORQ across the three samples are reported in Table 3.
Explained common variance is the proportion of all common variance for all items explained by a factor. In the case of the student and lay counsellor samples, Table 3 indicates that the general factor (fortitude) explained 44% and 57%, respectively, of the common variance. The specific factors (self-appraisal, support-appraisals and family-appraisals), therefore, explained 56% and 43% of the variance in the student and lay counsellor samples, respectively. This result confirms the multidimensionality of the FORQ for these samples, as the specific factors accounted for sufficient variance after the variance attributable to the general factor was taken into consideration. The omega/omegaS coefficient, which is a model-based estimate of reliability, further confirmed that  the self-appraisal (omegaS: students = 0.82; lay counsellors = 0.74), support-appraisal (omegaS: students = 0.78; lay counsellors = 0.85) and family-appraisal (omegaS: students = 0.77; lay counsellors = 0.89) subscales demonstrated sufficient reliability. It also confirmed the reliability of the general factor (omega = 0.88 and 0.91). OmegaHS, which is the reliable variance of the subscales after removing the variance attributable to the general factor, was reasonable in both the lay counsellor (self-appraisals = 0.54, support-appraisals = 0.10, family-appraisals = 0.43) and student sample (selfappraisals = 0.45, support-appraisals = 0.53, family-appraisals = 0.30).
Additionally, the results of the bifactor analysis with respect to the adolescent sample suggest that the FORQ is essentially unidimensional in this sample. First, the general factor explained 77% of the common variance, whilst the specific factors explained only 23% of the variance. Second, the model-based indicator of reliability, omegaS, reflected low levels of reliability for the self-appraisal (0.42) and supportappraisal (0.52) subscales. Third, omegaH, which reflects the percentage of variance in total scores attributable to the general factor, was very close to the cut-off point suggested in the literature. Reise, Bonifay and Haviland (2013) proposed that when omegaH is greater than 0.80, the scale can be considered essentially unidimensional. Lastly, omegaHS, which indicates the percentage of reliable variance of the subscales after considering variability because of the general factor, was extremely low in the three subscales (self-appraisals = 0.05; support-appraisals = 0.01; family-appraisals = 0.04).

Discussion
Given the different structures in which the FORQ has been used in published research, this article sought to investigate, through CFA and bifactor statistical indices, the factor structure and dimensionality of the FORQ in three samples: adolescents, students and adults. The CFA demonstrated that, in the student and lay counsellor samples, the bifactor model is a better fit compared to the one-factor model and only marginally a better fit than the three-factor second-order model. However, both the bifactor model and the second-order model confirmed the conceptualisation of the FORQ as consisting of a total scale and three subscales in these two samples. For the adolescent sample, there were no significant differences in indices of fit amongst the one-factor, the second-order model and bifactor models.
The standardised solution resulting from CFA suggested that only the general factor was meaningful in the adolescent sample, as only the loadings for the total scale were  significant, whilst the loadings on the specific factors were non-significant. In the case of the student and lay counsellor samples, the loadings on both the general and specific factors were significant, with the exception of two items loading on the self-appraisal and support-appraisal subscales in the case of the lay counsellor sample.
The bifactor statistical indices suggest that the FORQ was multidimensional in both the student and lay counsellor samples. The ECV indicates that the general factor only accounted for 44% and 57% of the variance amongst items in the student and lay counsellor samples, respectively, whilst 46% and 53% of the variance was accounted for by the subscales. The factor loadings confirm this result, as all the loadings, with two exceptions, were significant.
For the adolescent sample, the non-significant factor loadings and the bifactor statistical indices suggest that the FORQ in this sample is essentially unidimensional, as the general factor explained 77% of the common variance, whilst the specific factors explained only 23%. The model-based estimates of reliability were also very low for two of the subscales in this sample. It therefore appears that whilst the total scale (i.e., fortitude) is meaningful in this sample, the use of subscales may not be justified. This finding may be ascribed to the ongoing process of self-concept clarification that takes place during the adolescent phase of development and ultimately leads to the consolidation of appraisals of self, family and significant others.
Self-concept clarity refers to the extent to which beliefs about the self and others are clearly defined, internally consistent and stable across time (Crocetti, Rubini, Branje, Koot, & Meeus, 2016). A core task in adolescence is the acquisition of an enduring self-concept, which occurs through interactions with peers, parents and significant others (Laursen & Hartl, 2013). During adolescence, more time is spent with peers than family members; therefore, the reference group for social experiences and social support shifts towards the peer group. Indeed, peer acceptance has a significant impact on self-esteem and related appraisals of competence and worth (Crocetti et al., 2016). The adolescent phase is also associated with increased conflict in the parent-child relationship, possibly owing to the adolescent's increasing need for independence and autonomy. This conflict can lead to physical and emotional distancing from parents and other family members and can influence appraisals of the family environment and the family as a potential source of support (Moed et al., 2015).
The search for identity can lead to adolescents seeking out new experiences, taking on different roles and forming new relationships. Some of these relationships may be dissolved within a short period of time owing to new interests (Laursen & Hartl, 2013). Social perspective-taking abilities also increase, and adolescents come to better understand the extent to which others can be relied upon in times of need (Raufelder, Sahabandu, Martínez, & Escobar, 2015). As cognitive maturation is a gradual process, the adolescents in the study may have been in the process of consolidating their appraisals of self and others, which may account for the unidimensional nature of the FORQ in relation to the adolescent sample. Taken together, the CFA and the bifactor analyses seem to suggest that caution needs to be applied in using the FORQ subscales with children and adolescents. However, further research that replicates this finding in adolescents and other samples is needed before a definite conclusion about the suitability of the FORQ in different age groups can be reached.
Finally, the article demonstrates the importance of going beyond overall model fit statistics, especially with regard to bifactor models, and implementing bifactor analyses. Bornovalova et al. (2020) cautioned against the overreliance on overall model fit indices and highlighted the problem of overfitting in the case of the bifactor model. They argued that overall fit statistics favour the bifactor model over other models, and because of the flexibility of this model, the bifactor model 'can exhibit good global fit even if the pattern of loadings does not resemble a bifactor structure in any meaningful sense' (p. 2).

Limitations
The small sample size with respect to the lay counsellors should be considered a limitation, and it has affected the interpretation of incremental fit indices. Whilst it is possible that the FORQ is multidimensional for older age groups, this might just be a sample-specific result with a sample size of N = 146, and as pointed out earlier, this finding would need to be replicated several times before a definite conclusion can be made. Similarly, there is also evidence in younger population groups that most of the variance might be attributable to the general factor, and this needs to be replicated as the results for the different language groups were pooled. However, whilst age and language were offered as potentially explaining the unidimensionality of the scale in adolescents, there are many other potentially confounding variables that could have played a role. For example, the adolescents were drawn from a marginalised and extremely disadvantaged community; thus, socio-economic status might also have been a confounding variable.

Conclusion
This article confirms the multidimensionality of the FORQ and provides support for its use as a scale consisting of a general factor and three specific factors for young adults and adults. The results indicate, however, that for adolescents the FORQ is essentially unidimensional, which suggests that only the general factor should be used. However, further research is called for to replicate these tentative findings. The article also highlights the importance of going beyond overall model fit statistics, especially for bifactor models and to implement bifactor analyses to draw conclusions about the unidimensionality or multidimensionality of scales.