Psychometric Properties of the Job Engagement Scale: A Cross-Country Analysis

Job engagement is a motivational construct that refers to the willingness of employees to invest their physical, emotional and cognitive energies in their jobs in a holistic and simultaneous manner. Researchers use the Job Engagement Scale (JES) to measure the above conceptualization of job engagement, whose application is recent in job engagement research and is based largely on Western samples. In order to examine how job engagement is perceived in Asian contexts, this exploratory study aimed to provide a cross-country analysis of psychometric properties of the JES. We utilized data from earlier research of the first author, which were collected from 347 Pakistani and 498 Malaysian employees worked at diverse organizations. Psychometric analyses with reliability and validity estimations were performed using the Structural Equation Modeling. Results showed good internal consistency reliability, convergent validity and factorial validity of the JES for both Pakistani and Malaysian samples. However, psychometric properties of the JES for Pakistan outperformed those for Malaysia in all the estimations. Implications for future use of the JES and limitations of the study are discussed.


Introduction
Job engagement is a motivational construct that refers to the willingness of employees to fully invest their physical, emotional and cognitive energies in performing their jobs in a comprehensive and simultaneous manner (Kahn, 1990).Researchers and practitioners believe that engaged workforce is a key source of competitive advantage, because workforce possessing high level of job engagement is a unique resource that is rare, inimitable and non-substitutable (Albrecht, Bakker, Gruman, Macey, & Saks, 2015).However, the situation on ground shows that organizations seriously lack engaged workforce.In various global workforce surveys, analysts report that organizations across the globe are experiencing a serious decline in employee job engagement.According to Tower Watson's report (2014), only 40% of employees demonstrate job engagement with their work across the globe.Making the situation more alarming, Gallup (2016) recent study has found that only 13% of the employees are engaged in organizations worldwide, indicating that the world has a serious job engagement crisis with lasting consequences for the world economies.
Recognizing the importance of job engagement for the success of organizations, researchers and consultancy firms have developed different measures of job engagement to capture its essence (Albrecht et al., 2015;Harter, Schmidt, & Hayes, 2002).In terms of rigor, however, the job engagement measures developed by consultancy firms, such as Gallup Q12, have been criticized for lacking precision in measuring job engagement (Schaufeli & Bakker, 2010).Considering the limitations of these measures, researchers have clarified the meaning and measurement of job engagement and developed the scales having good psychometric properties.Among these scales, the Utrecht Work Engagement Scale (UWES) developed by Schaufeli, Salanova, González-Romá, and Bakker (2002) and the Job Engagement Scale (JES) recently developed by Rich, Lepine, and Crawford (2010) are two key scales being used in the job engagement research.The UWES is being used widely in the engagement research because of its sound psychometric properties across different cultures and its availability to researchers for 15 years.In comparison, the JES has received less attention for being relatively new (Rich et al., 2010).For this reason, application of the JES in engagement literature is limited in general and rare in Asian contexts.
The purpose of this exploratory study is thus to provide a crosscountry analysis of the psychometric properties of the JES on diverse samples of employees from two Asian countries, including Pakistan and Malaysia.Earlier studies utilizing the JES on Western samples have shown good psychometric properties of this scale and supported its three-factor structure (e.g., Alfes, Shantz, Truss, & Soane, 2012;Basit & Arshad, 2016;Byrne, Peters, & Weston, 2016;Chen, Yen, & Tsai, 2014;He, Zhu, & Zheng, 2014;Rich et al., 2010;Shuck, Twyford, Reio, & Shuck, 2014).However, application of the JES in Asia is relatively limited.This study will contribute to the job engagement literature by providing psychometric details of the JES in the Asian context and will facilitate to generate further research on job engagement in these important but underrepresented regions of the world.

Psychometrics Analysis
Analysis of psychometric properties of a scale is the first step to establish its scientific utility (Nunnally & Bernstein, 1994).Researchers use different statistical techniques for this purpose.At the basic level, psychometric analysis of a scale involves analyses of (1) internal consistency reliability, (2) convergent validity, and (3) factorial validity (Balducci, Fraccaroli, & Schaufeli, 2010).

The Utrecht Work Engagement Scale
Among several measures of engagement developed by researchers, the Utrecht Work Engagement Scale (UWES) of Schaufeli et al. (2002) is the most widely used scale in engagement research.This scale measures engagement that is defined as a positive work-related state of mind involving vigor, dedication, and absorption in work (Schaufeli et al., 2002).According to these researchers, vigor is willingness to put in energy while working and persistence in difficult situations; dedication is a sense of enthusiasm and pride; and absorption is fully concentrating in work.This conceptualization of engagement is rooted in job burnout and employee well-being theories.The studies utilizing the UWES on various Western and non-Western samples largely support its three-factor structure and also report that the three dimensions are closely interrelated and invariant across cultures and occupations (Schaufeli & Bakker, 2010).The 17-item UWES has been translated into more than 20 languages (Schaufeli & Bakker, 2010), and is available also in a short version of 9 items UWES-9; (Schaufeli, Bakker, & Salanova, 2006).

The Job Engagement Scale
After the UWES (Schaufeli et al., 2002), the second measure of engagement is the 18-item Job Engagement Scale (JES) developed by Rich et al. (2010).This scale is based on the landmark work of Kahn (1990) who defined engagement as the harnessing of employees' selves to their work roles by investing their physical, emotional and cognitive energies in simultaneous and holistic manner.According to Kahn, physical engagement refers to the extent of effort employees put in while performing their work roles; emotional engagement is the emotional involvement and feelings which the employees have about their work; and cognitive engagement is the mindfulness and mental attention of employees towards with work.This conceptualization of engagement is largely rooted in theories of motivation (Alderfer, 1972;Deci & Ryan, 1985), work design (Hackman & Oldham, 1980), and role performance (Goffman, 1961).At present, the original English version of the JES is not available in other languages.

Importance of the Job Engagement Scale in Research
Although the UWES has sound psychometric properties and is widely used in engagement research, it is not free from limitations.For instance, Saks and Gruman (2014) in their literature review have noted that as compared to Schaufeli et al. (2002)'s view, Kahn's (1990) definition of engagement is relatively precise and grounded better in theory.In a similar vein, Byrne et al. (2016) have examined the measures of engagement across five field samples and found that both the UWES and JES showed strong correlation but were not interchangeable.The authors concluded that because these scales measure different aspects of engagement, the UWES is more suitable in applied settings and the JES should be preferred in research settings.

Samples and Procedures
Pakistan.The Pakistani data were collected from two organizations operating in manufacturing and service sectors.The manufacturing organization was located in Faisalabad and ranked among the top producers and exporters of hosiery products.E-mail invitations were sent by HR manager to all the employees to participate in the online survey through the organization e-mail system.The e-mail contained the link of the survey and only the research team had access to the submitted responses.Out of 500 employees, a total of 210 participated in the survey and completed the survey on the same day during work hours, yielding a response rate of 42%.The service organization was Pakistan's largest commercial private bank.
Data were collected from its provincial headquarter located in Lahore during a half-day training session.On behalf of the author, a bank manager organizing that training shared the link of the online survey with 175 trainees and requested them to complete the survey at the end of the training.In total, 154 responses were received, yielding a response rate of 88%.After the preliminary analysis, only 137 were usable for further analysis.In total, 347 aggregated responses were used to analyze the psychometric properties of the JES on the data from Pakistan.
Table 1 presents descriptive statistics of the demographics variables of Pakistani respondents.Most of the respondents were male (75.5%), single (61.1%), and had earned a Master degree (67.7%).An average respondent was 31.2 years old (SD = 5.7) and was employed at the organization for 3.5 years (SD = 2.8).
Malaysia.Data from Malaysia were also collected from two organizations.The first organization was a large public university located in Selangor.Three hundred paper-and-pencil questionnaires were distributed among the academic and administrative staff of management, economics and engineering faculties.
In three weeks, 161 participants completed the questionnaires, yielding a response rate of 55%.The other organization was a public teaching hospital located near Kuala Lumpur.A research assistant distributed 500 paper-and-pencil questionnaires among the nurses in 26 wards.After two weeks, a total of 373 completed questionnaires were returned, yielding a response rate of almost 67%.As a result of preliminary analysis, only 337 responses were retained for further analysis.
In total, 498 aggregated responses were used to analyze the psychometric properties of the JES on the data from Malaysia.As shown in Table 1, analysis of variables of the Malaysian respondents revealed that most of them were females (86.3%), married (67.1%), and had earned diploma-level education (67.3%).Among various ethnic groups, most of the respondents were Malay (97.4%).The mean age of the respondents was 31.6 years (SD = 7.9) and the mean tenure was 8.7 years (SD = 7.5).

Measure
Job engagement was assessed using the JES (Rich et al., 2010).Kahn (1990) construct of engagement is multidimensional, therefore this scale consisted of three subscales to measure physical, emotional and cognitive engagement of employees.In total, there were 18 items in this scale with 6 items for each subscale.The respondents were asked to indicate their agreement or disagreement on a five-point Likert scale.

3.3.Psychometric Analyses
The SEM was used to examine the psychometric properties of the JES using AMOS package (Arbuckle, 2011).The Maximum Likelihood estimation method was applied on the covariance matrix of the items to generate parameter estimates (Bentler & Chou, 1987).
Goodness-of-Fit for the models was assessed using the Chi-Square, the Goodness-of-Fit Index (GFI), the Root Mean Square Error of Approximation (RMSEA), the Standardized Root Mean Residual (SRMR), the Tucker-Lewis Index (TLI), and the Comparative Fit Index (CFI).According to methodologists, model fit is attained when the RMSEA and SRMR are .08or less and the GFI, TLI, and CFI are .90or greater (Hair, Black, Babin, & Anderson, 2010).
Following Anderson and Gerbing (1988), the chi-square difference test was used for the comparison among different models.Two models are considered different if the value of this test is statistically significant.

Descriptive Analysis of the Job Engagement Scale Items
Table 2 presents the JES items and the related descriptive statistics.It is evident that both Pakistani and Malaysian respondents experienced all aspects of physical, emotional and cognitive engagement in their jobs.The mean values of all items for both the countries were above 4.0, indicating high level of job engagement experienced by Pakistani and Malaysian respondents.
Analysis of minimum-maximum values of 18 items of the JES showed that not a single Pakistani or Malaysian respondent responded with the minimum value of 1 (strongly disagree) for any item of physical engagement.The minimum values for this dimension were 2 for all items in the case of Pakistan and were 3 (neither disagree nor agree) for most items in the case of Malaysia.Few Pakistani respondents indicated minimum value of 1 for 4 items of emotional engagement, whereas the minimum value was 2 for this scale in the case of Malaysian respondents.
Finally, minimum value of 1 was reported by few Pakistanis for 3 items of the cognitive engagement and by few Malaysians for only 1 item of this scale.In sum, extreme lowest values for emotional and cognitive engagement were observed relatively more in number among Pakistani respondents as compared to the Malaysian respondents.
In order to determine normality, statistical software generally set the values of skew and kurtosis to zero for a normal distribution (Byrne, 2010;Hair et al., 2010).A distribution departs from normality when its skew and kurtosis are positive or negative.
According to DeCarlo (1997), when data is not normal, skew impacts the tests of means and kurtosis affects the tests of variance.There seems to be a lack of consensus over the clear cut threshold for deciding the extent to which departure from normality becomes a serious threat to the validity of the results.
According to Meyers, Gamst, and Guarino (2006), researchers may consider data to be sufficiently normal if the values of skew and kurtosis fall within the range from +1.0 to −1.0.In the case of covariancebased SEM, where larger sample sizes are usually required to produce reliable results, researchers recommend that the values of skew and kurtosis should be less than 2 and 7, respectively (Byrne, 2010;West, Finch, & Curran, 1995).Some researchers also suggest that non-normality has detrimental effects only in small samples and this effect diminishes effectively for the sample size of 200 or more (Hair et al., 2010;Tabachnick & Fidell, 2007).Thus, skew and kurtosis for both country samples in this study were moderate and did not affect the validity of the results presented here.

Internal Consistency Reliability and Correlations
The internal consistency reliabilities of the JES and correlations among its three subscales pertaining to Pakistani and Malaysian datasets are presented in Table 3.It can be noted that the internal consistency reliability for overall job engagement and its three subscales ranged from 0.89 to 0.97, which were well above the threshold of 0.70 (Nunnally & Bernstein, 1994) and, thus, indicated good reliability of the JES for Pakistan and Malaysia.
However, all reliability values of Pakistani sample (0.92-0.97) were greater than the Malaysian sample (0.89-0.95), indicating that the JES showed a relatively better reliability for Pakistan.A similar pattern can also be observed in correlations among three dimensions of job engagement.All correlations were strong for Pakistan as they range from 0.91-0.92,whereas correlations were moderate for Malaysia as they ranged from 0.66-0.73.
In sum, the results revealed that the JES demonstrated relatively higher reliability and strong correlations among the physical, emotional and cognitive engagement for Pakistan then it did for Malaysia.

Convergent Validity
In order to test for the convergent validity of the JES, factor loadings of each item were computed for both the countries.
Factor loading is a statistical estimate representing the relationship between a factor (latent construct) and its respective indicators (observed variables), and is generally interpreted in terms of a standardized regression coefficient (Kline, 2011).
Factor loading scores range from −1.0 -+1.0.According to Anderson and Gerbing (1988), the factor loading score of a measurement item should be greater than twice its standard error to make the factor loading significant.The most commonly used threshold for a factor loading is 0.70, and factor loadings above this value indicate high association between the factors and indicators (Hair et al., 2010).
Table 4 shows that factor loadings of all items of the JES on Pakistani data were above the threshold of 0.70 and significantly ranged from 0.72 to 0.86 (p <0.001).These results indicated that the JES achieved excellent convergent validity for Pakistan.In a similar vein, factor loadings of 16 items of the JES for Malaysia were above the threshold of 0.70 and ranged from 0.71 to 0.89 (p < 0.001).
However, only two items (item 6 and 7) were slightly below this cutoff but were above 0.60.These results indicated that the JES achieved sufficient convergent validity for Malaysia.

Factorial Validity of the Job Engagement Scale
In order to examine the factorial validity of the JES for both countries, values of the Chi-square and other fit indices of five CFA models were computed.As shown in Table 5, Model 1 and Model 3 were 3-factor models for Pakistan and Malaysia, respectively, in which all the JES items were loaded on their three respective latent constructs of job engagement dimensions.In a related vein, Model 2 and Model 4 were 1-factor models for Pakistan and Malaysia, respectively, where the estimated correlation parameters were constrained to 1.In order to assess whether a 3-factor or 1-factor model fit the data better, the Chi-square difference test was performed.
As indicated by the Goodness-of-Fit indices, Model 1 (3-factor, Pakistan) achieved good fit to data as all its indices satisfied the cutoff criteria.Model 2 (1-factor, Pakistan) showed poor fit to data as value of the GFI (0.83) was below the cutoff of 0.90, and values of the RMSEA (0.09) and SRMR (0.16) were greater than the cutoff value of .08. Surprisingly, Model 3 (3-factor, Malaysia) showed poor fit to data as values of the GFI (0.83), RMSEA (.10), TLI (0.88), and CFI (0.89) were slightly away from their required cutoff values.However, the fit was improved, as reflected in Model 3a, by freeing two error covariances of physical engagement (3, 6, 4 and 5) and two of emotional engagement (1 and 2, 4 and 5).Finally, Model 4 (1-factor, Malaysia) showed poor fit to data as values of the GFI (0.67), TLI (0.71), and CFI (0.75) were far below the cutoff value of 0.90, and values of the RMSEA (0.16) and SRMR (0.30) were greater than the cutoff value of 0.08.In addition, both 3-and 1-factor models for Pakistan and Malaysia were compared using the Chi-square difference test.Model 1 and Model 4 in comparison to Model 2 and Model 3a revealed that a 3-factor (physical, emotional, cognitive) model showed better fit to data than a 1-factor (overall engagement) model for both Pakistan and Malaysia, because the changes in the Chi-square difference test values of 3-factor models over 1-factor models were significant at p < 0.001.These results suggested that the original 3-factor structure of the JES was supported well for Pakistan.However, with some improvements, the original 3-factor structure of the JES also found support for Malaysia.

Discussion
In this study, psychometric properties of the JES were examined on the data obtained from 347 Pakistani and 498 Malaysian employees worked in diverse manufacturing and service organizations.Using various psychometric analysis techniques, this study is the first to perform a crosscountry analysis between Pakistan and Malaysia with regard to the reliability and validity of the JES.
There are four notable findings of this study with regard to psychometric properties of the JES.First, in line with earlier research showing internal consistency reliability of the JES ranged from 0.88 to 0.96 (e.g., Alfes et al., 2012;Basit & Arshad, 2016;Byrne et al., 2016;Chen et al., 2014;He et al., 2013;Rich et al., 2010;Shuck et al., 2014), the JES in the present study showed excellent internal consistency reliabilities well above the cutoff criteria of 0.70 (Nunnally & Bernstein, 1994) for both Pakistan (α = 0.97) and Malaysia (α = 0.95).This finding indicates that the perception of job engagement among our Asian employees were similar to the one perceived by Western employees.Thus, the JES is equally reliable in the Asian context.
Second, results of convergent validity showed that all items loaded on their respective job engagement factors relatively more strongly for Pakistan than Malaysia.For Malaysia, however, item 6 from physical engagement and item 1 from emotional engagement showed loading below the threshold of 0.70.In general, these results supported good convergent validity of the JES for both countries.Third, results of factorial validity assessed by the CFA for Pakistan and Malaysia showed that the three-factor (i.e., physical, emotional, and cognitive engagement) structure of the JES better fit the data in both countries than a one-factor (overall engagement) solution.These results are consistent with earlier research (e.g., Rich et al., 2010) and give support to the factorial validity of the three-dimensional conceptualization of Kahn's (1990) construct of job engagement among Asian employees.
Finally, analysis of modification indices showed that Malaysians interpreted items 3 and 6, and items 4 and 5 from physical engagement as very similar.In a related vein, Malaysians also interpreted items 1 and 2, and items 4 and 5 from emotional engagement as very similar.One possible reason for this difference might be the difference of languages between the two countries.
In our samples, English was the official language of work in Pakistani organizations, whereas Bahasa Melayu was the official language in the Malaysian organizations.We therefore recommend that only Malay version of the JES should be used in Malaysia to obtain high validity of this scale and to generate reliable results, particularly, when the respondents work in public organizations.Alternatively, researchers should consider removing the above-mentioned items from the JES because of high similarity in meaning.The main limitation of this study was the difference between the occupational samples of Pakistan and Malaysia.The Pakistani sample was from private organizations and the Malaysian sample was from public organizations.In addition, there were notable differences between the two countries in terms of sex, marital status, education and tenure.Thus, future research should take these differences into account to provide more stringent test of psychometric properties of the JES in cross-country analysis.
In conclusion, this study establishes the scientific utility of the JES in the Asian context of Pakistan and Malaysia.Psychometric properties of the JES were found to be good across Pakistani and Malaysian samples.However, estimates from Pakistan outperformed the estimates obtained from Malaysia.It is therefore recommended that either shorter English version of the JES to be used in the context of Malaysia by excluding items having greater similarity in meaning, or the JES version in Bahasa Melayu be used when the majority of respondents have Malay ethnicity or target respondents work in public organizations where Bahasa Melayu is the official language.