Introduction

The COVID-19 pandemic has ignited significant challenges for educational institutions around the world. Particularly challenging are the educational institution lockdowns (Ferrel & Ryan, 2020; Kapasia et al., 2020). Iran, as a COVID-19 hot spot, is no exception to this rule. On 23 February 2020, Iran's ministry of health announced the cessation of academic institutions and schools in numerous provinces (Taghrir et al., 2020).

During this period, e-learning, a direct result of the integration of technology and education, has surged as a powerful medium of learning, mainly through internet technologies (Al-Fraihat et al., 2020).The undeniable significance of e-learning in education has led to massive growth in the number of e-learning courses and systems offering different types of services. There has been a considerable surge in the usage of language applications, virtual training systems, video conferencing platforms, and online learning software packages since the outbreak of COVID-19 (Kapasia et al., 2020).

Online learning has been associated with an increase in behavioral, affective, and cognitive students' engagement in the online sector, and therefore the facilitation of this learning by academics to build a cohesive engagement with students is a crucial concern for universities.

Background

One of the biggest challenges faced by teachers in the classroom, globally, has been student non-engagement or dis-engagement (Fredricks, Filsecker, & Lawson, 2016). Engagement is defined as a positive state of well-being or high levels of energy; these are sometimes used interchangeably in the literature with involvement and commitment (Hallberg & Schaufeli, 2006) or belonging (Allen & Bowles, 2012). In an academic setting, student engagement refers to students' physical and psychological energy during educational experiences (Ribeiro et al., 2019).

Student engagement is fundamental to learning as it has potential effects on student academic performance and retention (Martínez et al., 2019). A growing body of literature recognizes the importance of students’ engagement in their academic outcomes. It has been found to have a mediating role in the relationship between transformational instructor-leadership and students’ academic performance (Balwant et al., 2019), students’ learning performance (Tao, Zhang, & Lai, 2018), student academic self-efficacy (course grades) (Papa, 2015), emotional intelligence, GPA and satisfaction with the university experience (She, Ma, Jan, Sharif Nia, & Rahmatpour, 2021; Zhoc et al., 2020) and lowed risk of burnout (Paloș et al., 2019; Rahmatpour et al. 2019a). The antecedents of student engagement comprise students having a sense of purpose, demonstrating persistence and resilience, reporting an emotional connection to others in the learning environment, feeling a sense of belonging to their place of learning, and reporting high self-efficacy (Payne, 2019).

Academic engagement, a core of this context, needs to be broadly defined as the ways and extent to which student is committed to or involved in online sessions proposed by an educational institution (Fredricks et al., 2016; Suárez-Orozco, Onaga, & de Lardemelle, 2010). In this context, the interaction between the student and the university is vital as emphasized by Bond et al. (2020) who found a positive relationship between the use of e-learning platforms and university student engagement (Bond et al., 2020).

According to Fredricks et al. (2016), there are different types of theoretical frameworks about engagement (Fredricks et al., 2016). For example, self-determination theory asserts that three psychological needs—autonomy, competence, and relatedness—enhance student engagement and intrinsic motivation (Badiozaman et al., 2019). On the other hand, control-value theory of achievement emotions explains the relationship between achievement emotions (enjoyment, boredom, anger, pride, and shame) and student engagement (Garn, Simonton, Dasingert, & Simonton, 2017).

The existence of different theories demonstrates that this concept is multi-dimensional and that student engagement is not only explained by students' behavior but also by their emotions and cognitions. Positive emotions (i.e., motivation, interest, et cetera) experienced by students during academic courses increase their desire to give greater effort and improve academic satisfaction, thus leading to greater engagement in learning (Rahmatpour, Sharif Nia, & Peyrovi, 2019b; Zhoc et al., 2020). Findings from the literature support the contention that cognitively engaged students have a tendency to enhance learning and effectively utilize learning resources to increase student engagement (Zhoc et al., 2020). In general, it could be concluded that student engagement significantly increases students’ willingness to participate in the learning process using certain behavioral, emotional, and cognitive processes.

Behavioral Engagement

Behavioral engagement is the carrier and representation of cognitive engagement and emotional engagement (D'Aniello, De Falco, Gaeta, & Lepore, 2020) and does not represent the student’s substantive participation. A student who demonstrates active behavioral engagement is most likely due to the student’s appearance and may not because of their cognitive and emotional engagement (Hu & Li, 2017). In this study, the dimension of behavioral engagement will be absorbed into the context of e-learning whereby there exist concerns in the involvement and contribution of academic assignments which is noticeable through students’ exertion, dedicating time, and persistence (Guthrie, Wigfield, & You, 2012; Latini et al., 2019). Previous studies have found that student engagement in online learning is the behavioral performance that includes reading course outlines and materials, asking questions, participating in ‘Blackboard’ interactive actions, and submitting assignment (Lee et al., 2015).

Cognitive Engagement

Cognitive engagement has traditionally been operationalized by measuring the amount of students’ assignment completion, attendance, extra-curricular activities, overall interactions with the lecturer, and how motivated they seem when engaging in classroom discussions (Appleton et al., 2008). Cognitive engagement refers to the extent to which students are able or willing to take on the learning assignment at hand, which includes the volume of effort students are willing to dedicate in working on the assignment (Hu & Li, 2017) and how long they are able to continue (Richardson & Newby, 2006). However, due to the lack of communication between academics and students, the performance of students who take part in online learning is not satisfactory, and their persistence and productivity is also poor (Hu & Li, 2017). Rotgans and Schmidt (2011) quantified that with students who individually search for data (i.e., online resources)—that is, students who engage in “self-initiated information-seeking behaviors”—the volume of autonomy would be relatively high and this consequently leads to more cognitive engagement. Their findings also found that involvement in a group assignment and engaging in discussions will possibly lead to either high or low sentiments of autonomy, reliant on the group dynamics. For instance, if there is a domineering peer in a group, students can experience less autonomy and engage less cognitively as opposed to a group where students collaborate well with each other (Rotgans & Schmidt, 2011). In an e-learning context, this study proposes that the degree of autonomy is essentially related to an activity or assignment and largely verifies the level to which students engage cognitively with that activity or mission of an online assignment.

Emotional Engagement

Emotional engagement is used as an internal state that provides the impetus to contribute in certain academic behaviors (Molinillo et al., 2018). In addition, emotional engagement is employed when the educators promote positive emotion in their students (Palmer, 2017). In this regard, prior studies indicated that emotional engagement could take place once both utilitarian needs and hedonistic activities are satisfied. For instance, Molinillo et al. (2018) acknowledged functional and hedonistic values as antecedents of emotional value, whereby the role of emotional value is intrinsic to satisfaction. They have suggested that emotional value is critical to understanding the individual experience and in enhancing overall satisfaction (Molinillo et al., 2018). To the authors’ knowledge, there have been limited studies done on emotional engagement in the e-learning context. Therefore, there is a need to examine emotional engagement through academic activities and the consequences of emotional engagement on students’ engagement and achievement.

In e-learning contexts, learners’ behavioral engagement is difficult to define clearly and cannot fully address “to the students’ efforts, so this study should consider students’ perception, regulation and emotional support in their learning process, including both quantity of engagement and quality of engagement, both communication with others and learning consciously, both guidance and help of others and self-management and self-control” (Hu & Li, 2017).

Due to the multidimensional nature of the student engagement concept (Fredricks et al., 2016), different scales with various subscales have been found to measure university students’ engagement. These scales were developed in different contexts based on specific educational systems. A sample of these scales is shown in Table 1.

Table 1 Some of the most common university student engagement scales and their factors
Full size table

Among these scales, the USEI, developed by Maroco et al. (2016) is solidly grounded in the self-determination theory and targets engagement to the university context; it is comprehensive and has fewer items than other scales. This scale has been assessed for measurement invariance across gender and student engagement in the university context and has gathered evidence of validity and reliability from a diverse set of different countries from Mozambique to Finland, Taiwan to the USA, etc. (Assuncao et al., 2020). The USEI scale consist of three domains: “cognitive” which refers to students' desire and effort to understand and master complex content and skills, “behavioral” which reflects students’ participation in classroom tasks and activities, and “emotional” defined in terms of students’ positive and negative reactions to classroom and schoolwork, as well as a sense of belonging in school (Maroco, Maroco, Campos, & Fredricks, 2016).

The literature examining students’ engagement and its contribution to academic outcomes in Iran (Mahdiuon et al., 2020; Ghanizadeh et al., 2020; Monavvarifard et al., 2019) is rather limited. What led the researchers to examine the items of USEI in the Iranian student's sample was that Iran had a new experience in online education during the COVID-19 pandemic, and before that, face-to-face and conventional education were widely used in all universities. For this reason, after the beginning of the COVID-19 pandemic and the forced use of online education, the necessary conditions and infrastructure were not sufficiently provided in Iran. It can be said that education and student engagement in online learning have undoubtedly been affected by this situation. Regarding the importance of student engagement in their' academic achievements, filling a gap in an e-learning context, the lack of valid and reliable scales based on Iranian university students' context, and the feasibility and suitability of USEI, this study aims to translate the USEI into the Persian language (P-USEI), and evaluate the psychometric properties of P-USEI among Iranian university students.

Methods

This methodological study was conducted among Iranian university students from April to May 2020. The protocol of this study was approved by the Mazandran University of Medical Sciences Research Ethics Committee (CODE: IR.MAZUMS.REC.1400.218).

Study Sample

The minimum sample size for conducting a factor analysis is equal to between 5 and 10 participants per item of the intended instrument (Kellar S. P & Kelvin E. A, 2012). The inclusion criteria for participants required that individuals be willing to participate and have online classes during the COVID-19 pandemic. A Google form of questionnaire was created, and the URL link was sent to social media groups (Telegram or WhatsApp) of university students. Students were selected through a convenience sampling. In total, the link was sent to 800 university students and 667 students in medical sciences (medical, nursing, midwifery) were filled the form (response rate: 83%). Only fully completed questionnaires were used in the data analysis.

Measures and Adaptation

The questionnaire composed of socio-demographic questions (i.e., age, gender, and marital status), educational-related information (i.e., type of university, the field of study, and GPA), and the University Student Engagement Inventory (USEI).

The USEI was developed by Maroco et al. (2016) for Portuguese university students and was recently validated for English, Slovakian, Italian, and Chinese languages (Assunção et al., 2020). It consists of 15 items and 3 factors, including behavioral (5 items), emotional (5 items), and cognitive engagement (5 items). The USEI is scored on a 5-point Likert-type scale from 1 (never) to 5 (always), and a reversed scoring method was used for one negative question (item 6: "I don't feel very accomplished at this school"). The range of scores for each of the subscales was between 5 and 25. Higher scores indicated higher student engagement (Maroco et al., 2016).

The World Health Organization's (2016) protocol of the forward–backward translation technique was applied to translate the scale from English into Persian. Certain words in the questionnaire were changed due to the contribution of a Persian scale to e-learning context. For instance, item 8 “I like being at school” was changed to “I like being at online class” whereby the word “class” was changed to “online class”, and the word “school” was changed to “online classes”.

Content Validity

Content Validity Ratio (CVR) and Content Validity Index (CVI) evaluated the items’ necessity and relevancy by 10 faculty members (3 professors, 2 associate professor, and 5 assistant professor). They were experts in medical education and have several studies in this fields. When the number of experts is 10, the minimum acceptable CVR based on Lawshe's (1975) table is equal to 0.62 (Lawshe, 1975). The minimum sufficient value for CVI for each item is 0.7.

Descriptive Statistics and Item Sensitivity

Descriptive statistics (mean, sd, min, max, skewness, and kurtosis) as well as item histograms were obtained for each item using the skimr package v. 1.0.5 (McNamara, Arino de la Rubia, Zhu, Ellis, & Quinn, 2018) for the R Statistical System (v. 4.0; R Core Team, 20).

Construct Validity Evidence

Construct validity was based on classical test theory and factor analysis framework. A confirmatory factor analysis (CFA) was conducted using the Weighted Least Squares Means and Variances (WLSMV) method and the most common goodness of fit indices, such as: Root Mean Square of Error of Approximation (RMSEA), Square Root Standardized Mean Residual (SRMR), Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), and the Normed Fit Index (NFI). RMSEA and SRMR below 0.08 and CFI, TLI and NFI above 0.90 were indicative of acceptable model fit (Hu & Bentler, 1999; Marôco, 2021). The scaled versions of the indices, which are appropriate for WLSMV estimation on the polychoric correlation matrix, were used. CFA were performed with the lavaan package v. 0.6.4 (Rosseel, 2012) for the R Statistical System. To improve the model fit, modification indices (MI) were considered and error correlations with MI larger than 11 (p < 0.001) (Marôco, 2021) were added when theoretical content and construct similarity were defendable.

Convergent and Divergent Validity Assessment

The convergent (the correlation between items of single subscale) and divergent validity (the extent to which factors are distinct) of the scale were estimated using Fornell and Larcker's approach (Fornell & Larcker, 1981) and the HTMT approach (Henseler et al., 2015).

The Average Variance Extracted (AVE) and Heterotrait-Monotrait ratio of correlations (HTMT) were estimated to assess the convergent and discriminant validity of the extracted factors. AVE larger than 0.5 are evidence of convergent validity. AVE larger than the squared correlations between factors and HTMT correlations smaller than 0.85 are indicative of discriminant validity (Henseler et al., 2015).

Reliability Assessment

Internal consistency was assessed by the ordinal Cronbach's alpha, McDonald's omega, and Average Inter-item Correlation (AIC). ωL1 and ωL2 estimates were calculated for the second order engagement factor. Coefficients and α values greater than 0.7 were acceptable (Mayers, 2013; McDonald, 1999). The AIC value between 0.2 and 0.4 indicated good internal consistency (Mohammadbeigi A, Mohammadsalehi N, & Aligol M, 2015). α and ω coefficients were calculated using the semTools package v. 0.5.1 (Jorgensen, Pornprasertmanit, Schoemann, & Rosseel, 2018). In addition, the Composite Reliability (C.R.) of subscales were calculated and the acceptable value is greater than 0.7.

Analysis of Invariance for Sex

Men and women have different goals when pursuing higher education in Iran. Thus, to measure student engagement in both sexes, the invariance of the measurement model needed to be assessed. A set of constrained models starting with a configural model were fitted with two sex (males and females) imposing equality constraints for factor loadings (metric invariance), intercepts (scalar invariance), factor means (means invariance), and residuals (strict invariance) using the equaltestMI package v.0.6.1 (Jiang & Mai, 2021) for the R Statistical System. ∆χ2 tests as well as ∆CFI and ∆DRMSEA were used to probe invariance. Evidence of invariance was accepted for non-significant ∆χ2 as well as absolute values of ∆CFI and ∆RMSEA smaller than 0.01 and 0.02, respectively (Cheung & Rensvold, 2002; Yuan & Chan, 2016).

Results

In this study, most students were female (69.3%) and single (88.3%). The mean age and GPA of participants were 22.53 years old (SD = 5.6) and 16.63 of 20 (SD = 1.4), respectively. More than half of students were studying at private universities.

Item psychometric Sensitivity and Content Validity

The items of P-USEI had acceptable CVI and CVR values. Item scores range, skewness, kurtosis, and CVI and CVR results are provided in Table 2.

Table 2 Distributional properties and CVI/CVR results of P-USEI’s items
Full size table

Construct-Related Validity

The CFA on the polychoric correlation matrix showed an overall good fit of the second order USEI tri-factorial model in the study sample (Fig. 1). Given the similarity of item contents and their presence in the same factor, as well as a second order latent structure that manifest itself on the first order constructs, and thus explains items' correlations both inter-factors and intra-factors, items were allowed to be correlated when modification indices were larger than 11 (p < 0.001). Thus items Q2 and Q3 from the behavioral factor; items Q8 and Q9 from the emotional factor; and items Q14 and Q14 from the cognitive dimension were correlated. The overall model fit to the data, with these three correlated items, was very good (χ2(84) = 470.825, Pp < 0.001, CFI = 0.971, NFI = 0.964, TLI = 0.964; SRMR = 0.046, RMSEA = 0.083 CI90% [0.076 to 0.091]). Table 3 gives the standardized factor loadings and statistical significance. None of the items had loadings below 0.4 and all reached statistical significance for p < 0.001.

Fig. 1
figure 1

The model of confirmatory factor analysis of the P-USEI

Full size image
Table 3 Convergent and discriminant validity assessment of P- USEI
Full size table

Convergent and Discriminant-Related Validity

Average Variance Extracted was 0.574, 0.705 and 0.675, respectively, for behavioral, emotional and cognitive engagement. None of the factor pairs had squared correlations larger than the AVE (see Table 3). Thus, both convergent and discriminant evidence of validity were observed in the study sample. The heterotrait-monotrait ratio of correlations ranged from 0.563 (emotional and behavioral engagement) to 0.661 (cognitive and behavioral engagement). Thus the HTMT method supports the evidence of discriminant validity of the USEI dimensions.

Reliability

Estimates for ordinal Cronbach α and McDonald’s ω are given in Table 4. In addition, the composite reliability (CR) of the three subscales ranged from 0.836 (behavioral e.) to 0.877 (Cognitive e.). The data displayed good reliability (See Table 4).

Table 4 Internal consistency and reliability of P-USEI
Full size table

Model Invariance for Sex

Proceeding from a configural tri-factorial model for both sexes (χ2 (168) = 440.967, CFI = 0.948, RMSEA = 0.071), the invariance analysis showed equal factor loadings for both sexes (∆χ2 (12) = 20.138, p = 0.065, ∆CFI = 0.002, ∆RMSEA = 0.001). Scalar invariance was observed using the ∆CFI (0.008), ∆RMSEA (0.002) criteria but not the ∆χ2 criterium (∆χ2 (12) = 52.712, p < 0.001). Strong mean invariance (∆χ2 (3) = 7.874, p = 0.049, ∆CFI = 0.001, ∆RMSEA = 0.000) as well as strict invariance (∆χ2 (15) = 28.995, p = 0.016, ∆CFI = 0.003, ∆RMSEA = 0.001) were also supported by the ∆CFI and ∆RMSEA criteria.

Discussion

The results of the present study support a three-factor structure of the P-USEI with 15 items with an overall student engagement second order factor. Similar to the original USEI (CR = 0.83, α = 0.74 to 0.88), the high level of Cronbach's alpha, McDonald's omega, and the correlation between the items demonstrated that three factors of the P-USEI had acceptable internal consistency. In addition, based on the results of CR, the P-USEI had good reliability. One of the advantages of measuring C.R. is that this estimate is not affected by the number of scale items and obtained structure and is dependent on the actual factor loading of each item of the latent variables (Vinzi, Chin, Henseler, & Wang, 2010).

The P-USEI had good convergent and discriminant validity. The USEI had discriminant validity, but convergent validity of the behavioral subscale could not be assumed in the original scale.

With the exception of item Q6r, all other items showed high individual reliability (factor loadings greater or equal to 0.7). Q6 is the only reversed item in the USEI. As observed with other validation studies (e.g., Assunção et al, 2020; Sinval, 2018) this reversed item always shows lower reliability as compared to the other USEI items. This may well reflect the profuse documented negative effect of item reversion on the item and scale’s psychometric properties (see e.g., Suárez-Alvarez et al., 2018; Zhang et al. 2016). Based on the CFA results, the P-USEI consists of three factors. The domains extracted from P-USEI were similar to the original inventory (Maroco et al., 2016) in which structure was confirmed with students across four continents (Assunção et al., 2020). The first factor, 'Cognitive', referred to university students' desire and efforts to acquire knowledge and solve their problems. This is consistent with the study done by Conduit et al. (2016) whereby they believed that cognitive engagement could be defined as activities that reflect the students' cognitive strategies and approaches to learning, such as self-regulated learning (Conduit et al., 2016). Glapaththi et al. (2019) mentioned that cognitive engagement has a positive influence on students’ learning and academic achievement (Glapaththi, Dissanayake, Welgama, Somachandara, & Sachinthana, 2019). In an e-learning context, students’ degree of autonomy indicates the level of students’ cognitive engagement, collaboration, and involvement in group discussions, searching around a task to obtain the relevant information on the internet, or contributing with an online session are likely to outcome in various levels of cognitive engagement. In this case, listening to a lecture is arguably the least cognitively engaging as it leads to little or no student autonomy (Kwon et al., 2014; Lee et al., 2015; Rotgans & Schmidt, 2011).

The second factor of the P-USEI is 'Emotional', which is related to the positive and negative feelings of students about school and schoolwork. Glapaththi et al. (2019) believed that emotional engagement is a student’s sense of belonging and valuing, the attitudes, interests, and connections to school that motivate students to do their school work (Glapaththi et al., 2019). Sinval et al (2018) defined the emotional dimension as positive and negative feelings and emotions (Sinval, Casanova, Marôco, & Almeida, 2018). In addition, Molinillo et al. (2018) has shown that students who are engaged with their work are more motivated and willing to interact with the subject content, which is reinforced through collaborative work mediated by computers (Molinillo et al., 2018). However, in online collaborative assignments, the commitment may be even greater as it allocates for more flexible communication and eliminates the concern of physical interaction, forming a brighter preparation and managing rules (Ituma, 2011; Robinson, 2013). Many studies have proven that engagement has a positive influence on effective learning (Hämäläinen & Vähäsantanen, 2011; Ituma, 2011). Moreover, emotional engagement is able to lead an impetus of student engagement in an online environment, which will have a direct or indirect impact on behavioral engagement and cognitive engagement in the e-learning process. Consequently, they concluded that students’ positive emotions will stimulate their use of the knowledge and take effective strategies to complete the learning task, and the completion of the current learning assignment will stimulate the enthusiasm and interest to complete the assignment (Hu & Li, 2017).

The last factor extracted is 'Behavioral', and it referred to student participation in schoolwork. In Gundogdu and Asan's study, they stated that behavioral engagement is a kind of participation, observable behaviors, and performance of learners (Gündoğdu & Asan, 2019). Also, students' involvement encourages students to prepare more thoroughly for class, leading to improvements in exam scores and academic achievement (Paul T Balwant, Kamal Birdi, Ute Stephan, & Anna Topakas, 2019). Prior studies indicated that if students show active cognitive engagement and emotional engagement, they must show active behavioral engagement too (Hu & Li, 2017). In this regard, research has indicated that negative behavioral engagement, their cognitive engagement, and emotional engagement must not be high.

The present study focused on Iranian medical sciences students so that may limit the generalizability of findings. The self-report method of the survey may have led to some errors. Another limitation is the inclusion criteria for participants that required individuals to have online classes during the COVID-19 pandemic. P-USEI can explore the period in which students have no pressure for applying for online learning or participation in e-learning sessions.

This study adds to the large body of evidence of the validity and reliability of data on student engagement across different countries and socioeconomic and cultural contexts as diverse as Europe, Asia, Africa, or South America. The Persian version of the USEI can help researchers and university managers develop models, describe student university engagement, and predict the outcomes of student engagement. More broadly, it will allow educators to have more knowledge about the cognitive, emotional, and behavioral engagement of students, thereby making a difference to the students' educational experience and increasing student academic achievement and satisfaction.

Conclusions

The results of psychometric data analyses support the tri-factorial P-USEI among Iranian university students. The valid and reliable P-USEI could be useful for measuring Iranian university student engagement. The findings suggest that researchers should consider the effect of different student engagement in learning and academic achievement. The configuration of P-USEI dimensions, as shining candles and collecting data via new learning instruments, can be generalized by developing the research through a psychometric evaluation on student engagement in cross-cultural studies.