Assessing construct validity of the Grit-S in Chinese employees

This research examined the psychometric properties and construct validity of the Short Grit Scale (Grit-S) in Chinese insurance employees (N = 2,363; 37% males; mean age = 35.14). Exploratory factor analysis and confirmatory factor analysis (CFA) were used to determine the factor structure of the Grit-S. The resulting model was tested by multi-group CFA for the factorial invariance of the Grit-S across genders and age groups. Results showed that the Grit-S could be best explained by a two-factor model containing consistency of interest (α = .70) and perseverance of effort (α = .75). The factor model was equivalent across genders and age groups. The scores of the Grit-S were significantly correlated with external criteria variables including mental wellbeing and job performance. Overall, our findings suggested that the Grit-S can be a promising assessment of the grit trait in Chinese employees.


Introduction
Grit was initially proposed as the trait of strenuously sustaining ambition regardless of failure or adversity. It requires that an individual show persevering effort and sustaining consistent interest [1]. The concept is different from cognitive ability, but essential to success. Grit is found to be a valid construct in both adolescents and adults in various domains [1], such as academic [2] or professional [3].
To measure grit, Duckworth and colleagues (2007) [1] developed a 12-item scale based on the theory that grit entails components of (1) perseverance of effort and (2) consistency of interest. The two-factor structure of the scale was supported in an exploratory factor analysis (EFA) [1]. Duckworth and Quinn revised this scale and reduced the scale to eight items, naming it Grit-S [4]. The Grit-S has a hierarchical model structure, with two factors of the Grit-S loaded on a second-order latent factor called grit [4].
A recent meta-analysis [5] pointed out that the hierarchical model structure of the Grit-S is problematic [5] as the second-order factor has only two first-order factors as indicators. The effects of the second-order factor on the first-order factors may not be identifiable without a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 imposing additional constraints, such as constraining the factor loadings of the two first-order factors equally onto the second-order factor. Such a hierarchical model becomes no different from a simple single-factor structure with two-correlated first-order factors.
Early examinations of the psychometric properties of the Grit-S used primarily American samples, most of whom were undergraduate students [4,6,7]. The findings on the validity and reliability of the Grit-S in samples from other cultural and language backgrounds were mixed. Studies that provided some support for the Grit-S covered samples which included Chinese high school students [8], German university students and high school students [9], Japanese university students [10], Polish adults (mostly college students) [11], and Turkish university students [12]. However, the two-factor structure of the Grit-S was not replicated in a Filipino sample [13]. Unfortunately, most of the samples in previous studies were of college students; as such, understanding about the construct of the Grit-S within the broader population is limited. Thus, studies with non-student samples are needed to extend the generalization of the grit instrument.
Previous studies demonstrated that the Grit-S measures grit invariantly across genders in both western [1,9,11] and Southeast Asian samples [14]. The measurement invariance (MI) enables the exclusion of any measurement artifact in cross-group comparisons such those between males and females. The MI needs to be satisfied before making comparisons across groups using the scores of the scales [15,16]. In a Chinese context, it is unclear whether the Grit-S is measuring the same construct in males and females.
With regards to the criterion validity, extensive studies have shown that grit has positive relationships with constructs including but not limited to self-efficacy [17,18], conscientiousness [4,14,19,20], life satisfaction [17,21], and self-control [8,22]. Meanwhile, grit has negative relationships with depression [18,23] and stress [24,25]. It has also been demonstrated that grit can be used to predict empirical or life outcomes. It is well-documented that grittier individuals attain higher levels of education [17], and express higher levels of workplace retention [3], less counterproductive work behaviors [18], and a better working performance [19,[25][26][27]. The recent meta-analysis of 73 studies (N = 66,807) [5] shows that the Grit-S is moderately correlated with both performance and retention. The perseverance of effort factor exhibits the strongest prediction on academic success. The authors recommended examining whether these findings hold true in different domains-such as the workplace-that involve a greater number and range of tasks with different levels of difficulty. To our knowledge, the criteria of empirical validity has not been extended to examine the insurance sector, where professional performance-including status of insurance premiums and commission charges-may contribute to mirror the relationship between grit and performance more explicitly.

Current Study
The present study aimed to examine the psychometric properties of Grit-S in a large sample of Chinese insurance agency employees. First, it examined the factor structure of the Grit-S by using EFA and CFA. We used half of the sample for an EFA to explore the structure of the Grit-S, then conducted CFA on the other half to examine the factor structure of the model generated from the EFA. We expected that the original two-factor model proposed by Duckworth and colleagues (2007) [1] would fit our sample well [8]. We then tested the MI of the Grit-S across gender and age groups.
Second, we examined the external validity of the Grit-S accounting for more relevant variables, particularly work-related constructs including psychological distress, burnout syndrome, as well as conflict between work and family. Previous studies revealed that psychological distress [28], burnout syndrome [29], and conflict between work and family [30] are all unfavorable situations in which individuals can achieve success, thus, we hypothesized that they are significantly negatively correlated with the total score on the Grit-S. In addition, we analyzed the relationship between the Grit-S scores and the professional performances of the participants (i.e., status of insurance premiums and commission amounts). The hypothesis was that an individual with high-level job preferences would achieve high Grit-S scores.

Participants and procedure
The participants were 2,363 insurance agency employees recruited from 39 insurance companies in Guangdong, China. The age of the respondents ranged from 19 to 70, and approximately 69% were less than 40 years old (mean age = 35.14, SD = 8.99). The descriptive statistics of the participants are summarized in Table 1.
All insurance agency employees were administrated at the same time by their companies during morning conferences. Participants took approximately 30 minutes to complete the questionnaire, following a standard procedure. Before participation, participants were informed about the rules and goals of the study, and told they could withdraw from the study at any time. All questionnaire items were written in Chinese. This study was approved by the Human Subjects Review Committee at Guangzhou University.

Measures
Grit-S. The Grit-S is a short-form scale of the GRIT scale, which was developed by Duckworth and Quinn (2009) [4]. The Chinese version was translated by the laboratory of Duckworth (http://angeladuckworth.com/research/). Half of the Grit-S was made up of positivelyworded items (e.g., "Setbacks don't discourage me."), and half were negatively-worded items (e.g., "I often set a goal but later choose to pursue a different one."). Respondents selected the options most suitable for themselves on a scale of 1 ("Not at all like me") to 5 ("Very much like me").

BSI.
The BSI (BSI-18, [31]) is a scale that measures the psychological distress that individuals have experienced in the previous seven days. This five-point Likert-like scale measures the severity of the physical and mental state of a person, from 0 ("Not at all") to 4 ("Extremely"). It has been shown to have satisfactory internal consistency test-retest reliability [32]. The BSI-18 has four factors: somatization, depression, panic, and general anxiety; the alpha coefficients of the current sample for were .87, .86, .88, and .83, respectively. C-MBI. The C-MBI was developed by Maslach and Jackson (1981) [33] to assess individual burnout syndrome via three components: emotional exhaustion, depersonalization, and lack of personal accomplishment [33][34][35]. Each item was scored on a seven-point scale, from 0 ("The feeling has never been experienced".) to 6 ("The feeling is experienced daily."). The C-MBI has good reliability in previous studies [36]. The alpha coefficients of the present study for emotional exhaustion, depersonalization, and lack of personal accomplishment were .81, .72, and .73, respectively, and these values indicated acceptable to good internal consistency.
Work-family and family-work conflict scales. The work-family and family-work conflict scale is a five-point Likert-like scale with 12 items representing two factors: work interfering with family, and family interfering with work. Each factor has three dimensions: psychological resource, emotional, and behavioral conflicts [37]. Participants rate options according to the frequency they experience the described situation (1 = "Occurs fairly rarely"; 5 = "occurs always"). In the current study, the alpha coefficients for "work interferes with family" and "family interferes with work" were .82 and .86, respectively.

Data analysis strategy
First, descriptive statistics of each Grit-S item were calculated. To determine the factor structure of grit, EFA was conducted on a random split-half sample and CFA was applied to the other sample half to find and confirm the optimal model to explain the data. The whole sample was used to test the MI across gender and age groups. Next, the group differences were examined by comparing the means of the overall grit scores. Finally, zero-order correlation was used to compute criterion validity, and the differences in grit score among the insurance employees were tested through one-way ANOVA. The descriptive statistics were calculated by SPSS (IBM, SPSS Version 19, 2010). EFA and CFA were conducted by Mplus 8.0 [38].
Stage 1: Factor structure. Robust weighted least squares with mean and variance adjustment (WLSMV) was a suitable method for the estimator because the data was categorical [39][40][41]; thus, EFA with the WLSMV estimator with oblique rotation was conducted to identify the factor structure of the Grit-S. Additionally, parallel analysis and a scree plot were used with the robust maximum likelihood estimator to determine the number of factors [42].
To confirm the structure generated from EFA, CFA was conducted using the WLSMV estimator. Fit indices, such as chi-squares, root mean square error of approximation (RMSEA), Tucker-Lewis index (TLI), and comparative fit index (CFI), were obtained to evaluate the goodness-of-fit of the model. An RMSEA value smaller than .08 represents an acceptable fit, whereas a value of .06 or lower indicates good fit [43,44]. TLI and CFI values higher than .90 indicate an acceptable fit, and values exceeding .95 represent an excellent fit [45].
Stage 2: MI and mean comparisons. MI tests were conducted across gender and age groups using multi-group CFA. Each group model generated from EFA and examined in CFA was initially assessed before conducting the MI test [46]. Next, four levels of MI tests (i.e., configural, metric, scalar invariance, and error variance) were carried out. The differences in fit indices between the unconstrained and constrained models were obtained to examine whether a specific level of MI was achieved. Because the chi-square test is sensitive to sample size and small differences yield considerable variations if the sample is too large [47,48], we considered the difference in CFI (ΔCFI) and TLI (ΔTLI) as suitable indicators of MI [49]. According to Cheung and Rensvold (2002) [49], the equivalence is acceptable when ΔCFI � .010 and ΔTLI � .010.
The mean comparisons between gender and age groups were then conducted. If Grit-S is invariant across gender and age groups, then a comparison between different gender and age groups is also valid. The mean difference of gender and age groups was analyzed via t-test and one-way ANOVA, respectively, using SPSS (IBM, SPSS Version 19, 2010). The statistical tests adopted a significance level of .05.

Descriptive statistics and factor structure
The descriptive statistics, including means, standard deviations, skewness and kurtosis, were summarized in Table 2. We used a random split-half sample (N = 1,181) to conduct EFA with the WLSMV estimator. This analysis yielded two factors with eigenvalues of more than 1.00 (2.84 and 1.61). Furthermore, the scree plot test and parallel analysis suggested that the twofactor model was suitable. Fig 1 shows the results of the parallel analysis in detail. Each item was loaded onto its target factor, and all factor loadings were significant at p < .001. In line with prior studies [4,10,12], the current Grit-S could be best explained by a two-factor model using consistency of interest (item: 1, 3, 5, 6) and perseverance of effort (item: 2, 4, 7, 8). The detailed statistics are summarized in Table 2. The correlation between the two factors was modest (r = .49, p < .001). The reliability analysis indicated that the alpha coefficients for the overall grit score, consistency of interest, and perseverance of effort were .85, .70, and .75, respectively.

MI and mean comparison
MI was performed across gender and age groups. Gender was divided into male (N = 865) and female (N = 1,481) groups, and age was divided into three groups: Lowest thru 29 (N = 709), 30-39 (N = 866), and 40 thru Highest (N = 705). Additional demographic characteristics are shown in Table 1.
The results of the MI are shown in Table 3, and two fit indices (i.e., CFI, TLI) indicated an acceptable fit of the two-factor model (CFIs > .95, TLIs > .95). Although significant Δχ 2 occurred when examining the invariance, other fit indices did not exceed the threshold of .01; this finding indicated that scale equivalence on gender and age for the four models (configural, metric, scalar, and error variance) was acceptable. Overall, the results of the MI indicated that Grit-S was equivalent across gender and age groups.
Results from the t-test and one-way ANOVA indicated that the mean grit score of male sample groups was not significantly different from that of the female groups, t (0.05/2, 2344) = 1.74, p > .05, d = .07. However, the mean grit score was significantly different across different age groups: F (2, 2285) = 3.34, p < .05,η 2 = .003. The post-hoc test suggested that only the difference between the age groups of below-29 and above-40 was significant (mean difference = -.706, p < .05).

Criteria and empirical validity
Results of the zero-order correlation analysis showed that the Grit-S had acceptable criterion validity in this investigation. The two-factor Grit-S total scores were significantly negatively correlated with BSI, C-MBI, and work-family and family-work conflict scores. The details of correlation and descriptive statistics among these variables are presented in Table 4. Furthermore, insurance employees with different levels of job performance showed different level scores on the Grit-S. The higher scores on the Grit-S correlated with higher group To further examine how well grit and psychological wellbeing (BSI) can predict job performance, mediation analysis with the WLSMV estimator was carried out. A significant mediation pathway was found between the BSI and job performance through the Grit-S (see Fig 4). The indirect effect of BSI on job performance is-.088, p < .001, and the ratio of intermediary effect to total effect is 47.06%, suggesting that the Grit-S substantially mediated the effect of BSI on job performance.
To compare the different functions of the two facets of grit, the difference in the magnitudes of their correlations with external variables were tested. As shown in Tables 4 and 5, the consistency of interest and perseverance of effort had significantly different correlations with all criteria variables except for BSI and commission amounts. The consistency of interest factor had stronger relationships with those variables than the perseverance of effort did.

Discussion
This study aimed to examine the psychometric properties and MI of the Grit-S. EFA and CFA indicated that the original two-factor model of the Grit-S fit the data well. In addition, the Grit-S was supported by the MI, as the means difference was not significant across gender, but was significant with a small effect size across age groups. The overall two-factor (i.e., perseverance of effort and consistency of interest) Grit-S exhibited significant correlations with external criteria. Furthermore, empirical validity was demonstrated by the significant relation between participant grit and job performance of participants.
The first aim of this work was to test the factor structure of the Grit-S in mainland Chinese adults. The results of EFA and CFA supported our hypothesis that the original two-factor structure fit our sample well. The structure also demonstrated MI across genders and age groups. These findings provide evidence for the applicability of the Grit-S in Chinese professional samples.
Next, we found that there were no significant differences between the scores of the two genders on the Grit-S. This is in line with previous findings in western samples [4,10,12].  However, significant score differences were found between the 19-29 age group and the 40-70 age group. A previous study had proposed that grit increases significantly with age [1], and our findings are partly consistent with this previous study, but the effect size is small. One possible reason for this is that we endorsed an age group classification (19-29, 30-39, and 40-70) different from that used in the previous study (25-34, 35-44, 45-54, 55-64, and 65 and above). Hence, we suggest that further investigations adopt older participants than those used in this study when comparing grit across different age levels.
The Grit-S showed criteria and empirical validity when used for mainland Chinese participants. In the current study, we selected variables more specifically related to work, including psychological distress, burnout syndrome, and conflicts between work and family. The Grit-S was significantly negatively correlated with these external variables. The negative correlations between grit and these psychopathological variables further examined the proposition that grit is a significant indicator of success and performance [1][2][3]. Insurance agents not only need to have a good knowledge about insurance offerings and regulations, but also must form relationships and build trust with customers. This process of relationship building requires both endurance and the ability to cope with rejection from customers, which are essential features of grit. Grit was correlated significantly with psychological wellbeing (BSI), indicating that individuals with high Grit-S scores were inclined to experience less somatization, depression, panic, anxiety, burnout syndrome, and work-family conflicts than those with low scores. We explored how grit and BSI predict job performance via mediation analysis and found that grit significantly explained the contribution of psychopathological factors (BSI) on job performance. This suggests that the negative impact of poor psychological wellbeing on job performance can be partially incurred by a lack of or decrease in grit.
Finally, we found that the consistency of interest factor showed significantly a stronger correlation with the criteria than perseverance of effort did, except for with BSI and participants' commission amounts. This result is not in line with previous findings, that perseverance of effort showed a stronger relationship with performance [5,13]. Consistency of interest is promoted widely in mainland China in colloquial forms, such as in sayings like, for example, "three days fishing, two days drying nets" or "constant dripping will wear away a stone." Therefore, mainland Chinese might think highly of consistency of interest when compared to people living in other nations. With regards to commission amounts, the complexity of the insurance occupation may demand the two factors (i.e., BSI and commission amounts) work together. However, the underlying functional mechanism of grit and its relation to professional criteria and performance has not been explored thoroughly yet. This therefore calls for the need to carry out mechanism investigations considering more contributing factors.
Some limitations existed in this current study. We selected a sample of Chinese insurance employees from around the same region of the country. Consequently, the findings may not be generalized to individuals in other areas of mainland China. Furthermore, the current study used only a Chinese sample, without the inclusion of samples from other cultural contexts. Thus, there is a lack of direct cultural comparison. It is unclear if the Grit-S measures grit in the same way across different cultures. Similarly, the profile of our sample allowed us to explore grit in a single professional domain. However, the use of only one profession may limit the generalizability of the results to individuals from other occupations. Therefore, further studies should extend investigations to different occupations. Another limitation is that all the items measuring consistency of interest are negatively-worded, and items measuring perseverance of effort are positively-worded, which also may have influenced the EFA and CFA in exploring the structure of grit [51]. A third limitation is the reliance on self-reported measures for the criteria and empirical validity analyses. Such a methodology introduces shared method variance, which can inflate the magnitude of observed correlations. The cross-sectional design  of our research is another methodological limitation that constrained observations of the development of external variables, such as BSI, C-MBI, the work-family and family-work conflict scales, as well as the performances of participants; therefore, a longitudinal design in future studies would be beneficial. In summary, our findings suggest that the Grit-S is a reliable and effective instrument when using with two factors (consistency of interest and perseverance of effort) for measuring grit in Chinese employees. The current study provides sound support for the MI in comparing gender and age group differences using the Grit-S. Furthermore, the Grit-S plays an important role in measuring people's psychological factors and performance.