Construct Validation of the Motivated Strategies for Learning Questionnaire in a Singapore High School Sample

In this study, the construct validity of the Motivated Strategies for Learning Questionnaire (MSLQ) was assessed. Participants were 441 Year 11 students in Singapore. Three separate confirmatory factor analyses were conducted for each section of the MSLQ (motivation and learning strategies). Results indicated that the original factor structures proposed by the instrument developers produced the best model fit. Cronbach α coefficients were also acceptable for all but one of the individual scales. Correlations with the Revised Learning Process Questionnaire—Two Factor and physics achievement scores also aligned with the theoretical basis of the MSLQ. These results confirmed the potential utility of this instrument for assessing the motivation and learning strategies of secondary students in Singapore.


Introduction
The 21st Century Competency Framework developed by the Singapore Ministry of Education in 2010 places central importance on the development of students' self-regulatory skills (Ministry of Education Singapore, 2010).The term self-regulation, when applied to learning, refers to the proactive process whereby learners set goals for their learning, actively monitor their progress, and regulate their cognition, motivation and behavior in order to achieve their learning goals (Pintrich, 2000).Research has indicated that individuals with higher levels of self-regulation are not only more successful in schools, but also in other aspects of their lives.In addition to performing better academically, these students tend also to achieve greater success in their careers, and enjoy better health, than those with lower levels of self-regulation (Bandura, 1982;Baumeister, Heatherton, & Tice, 1994;Boekaertz, Pintrich, & Zeidner, 2005;Locke & Latham, 2002;Mischel, Shoda, & Rodriguez, 1989).
The MSLQ is divided into two distinct sections.The motivation section assesses three main constructs (García & Pintrich, 1995): values (i.e., students' perceptions of the importance and interest of tasks), expectancy beliefs (i.e., students' beliefs about their task competency), and affect (i.e., students' emotional reactions to learning tasks).Six scales are used to assess these three constructs: Intrinsic Goal Orientation, Extrinsic Goal Orientation, Task Value, Control of Learning Beliefs, Self-efficacy for Learning and Performance, and Test Anxiety.Amongst the six motivation scales, Intrinsic Goal Orientation, Task Value, Control of Learning Beliefs and Self-efficacy for Learning and Performance are often regarded as "positive" motivations, as these have been linked to desirable education outcomes.Conversely, Extrinsic Goal Orientation and Test Anxiety are often linked to less desirable education outcomes, and hence are regarded as "negative" motivations.
The learning strategy section also assesses three main constructs (García & Pintrich, 1995): cognitive strategies (i.e., students' ways of processing information from reading materials and lessons), metacognitive strategies (i.e., students' control and regulation of their own thinking processes), and resource management (i.e., students' control and usage of learning resources).Nine learning strategies scales are used to assess these three constructs: Rehearsal, Elaboration, Meta-cognitive Self-regulation, Critical Thinking, Time and Study Environment, Effort Regulation, Peer Learning, and Help Seeking.In all, the MSLQ includes 81 items, to which students respond on a seven-point scale (not at all true of me to very true of me).Rehearsal is generally regarded as a "negative" strategy, as this has been linked to the approach of learning by rote.All other strategies are generally regarded as "positive" strategies, as these are generally linked to higher order thinking.Table 1 presents the overall structure of the MSLQ, along with sample item statements.The first validation study on the MSLQ was conducted by the instrument developers (Pintrich et al., 1991) with a sample of college students in the United States.Confirmatory Factor Analyses (CFAs) performed in this study indicated a five-factor structure for the motivation section, and a nine-factor structure for the learning strategies section.High levels of internal consistency were found for most scales.These findings have since been supported by studies conducted with other samples of college students in the United States (Cho & Summers, 2012), Oman (Alkharusi et al., 2012) and Singapore (Rotgans & Schmidt, 2010).
In comparison to the supporting evidence that has accumulated with regard to college students, little evidence has been published on the validity of the MSLQ at the secondary level.At the time of writing, only two published studies involving secondary students could be located.Erturan Ilker, Arslan and Demirhan (2014) and Karadeniz et al. (2008) both investigated the factor structure of a Turkish version of the MSLQ.The Karadeniz et al. study involved 1114 Turkish students from three primary schools and three secondary schools, while the Erturan Ilker et al. focused only on secondary level students.Both studies indicated that the instrument exhibited a similar factor structure to that found in earlier studies with college students.
Given the comprehensive nature of the MSLQ, this instrument has potential for monitoring the self-regulation levels of secondary students in Singapore.To date, however, no validation studies in this context could be located.Thus, in the present study, the validity of the MSLQ was examined using a Singapore secondary school sample.Two key aspects of validity evidence were examined within the study, based on the guidelines of the 2014 Standards for Educational and Psychological Testing (AERA, APA, & NCME, 2014).The internal structure of the MSLQ was first investigated by replicating the CFAs published in previous validations, and by assessing the internal consistencies of, and inter-correlations between, the MSLQ scales.Correlations with external variables which measure theoretically related constructs were then examined to further evaluate the construct validity of the instrument within this sample.

Sample
Participants were 441 Year 11 students (267 male, 174 female) within the Singapore secondary system (age M = 16.7 years, SD = 0.82).Participants were enrolled in a two-year physics course taught in the English language, which would prepare them for the General Certificate of Education Advanced Level (GCE "A" Level) physics examination, at the time of the study.As such, the MSLQ was administered in this study with specific reference to physics classes, as recommended by Pintrich et al. (1991).

Validation Instruments
In addition to the Motivated Strategies for Learning Questionnaire (MSLQ), two further instruments were used for validation purposes: the Two-factor Revised Learning Process Questionnaire (R-LPQ-2F; Kember et al., 2004), and a physics achievement test.The R-LPQ-2F includes two main scales (Deep Approach and Surface Approach), each of which includes a motive and strategy subscale (i.e., four subscales in all: Deep Motive, Deep Strategy, Surface Motive and Surface Strategy).Studies on the R-LPQ-2F have indicated that it demonstrates sound psychometric properties (e.g., Phan & Deo, 2007;Socha & Sigler, 2012).Drawing upon the theoretical bases of the two instruments (e.g., Biggs & Tang, 2007;Curran & Bowie, 1998), deep motive scores would be expected to correlate positively with the positive motivation scales in the MSLQ, while deep strategy scores should correlate positively with the positive MSLQ learning strategy scales.Surface motives would be expected to correlate negatively with the positive MSLQ motivation scales, and positively with the two negative MSLQ motivation scales, while surface strategies should correlate positively with the negative MSLQ strategy scale of Rehearsal.Expected correlations between the MSLQ and R-LPQ-2F are summarized in Tables 2 and 3. Given that both motivation and learning strategies are presumed to relate in some way to student achievement, a physics achievement test used within the school was used to provide further information on the validity of the MSLQ.This test is a two-hour pen-and-paper assessment comprising 15 multiple-choice questions and 3 short response questions, that are adapted directly from past year GCE "A" level examinations.The test questions, including the marking scheme, were vetted by the subject coordinator and the head of the physics department, who each had more than ten years of teaching experience, to ensure close alignment to the assessment objectives of the GCE "A" level physics examination (Singapore-Cambridge GCE "A" level physics syllabus 9646, 2014).Scores on the Physics Achievement Test were computed by summing the scores for the multiple-choice questions and the structured questions.The multiple-choice and structured questions were each worth a maximum score of 15 and 65 points, respectively, giving a total maximum score of 80. Expected correlations between the MSLQ scales and physics achievement, based on the underlying theory of the MSLQ, are shown in Tables 4 and 5. Permission was also granted by the Principal of the participating school.All participating students took the R-LPQ-2F, the MSLQ and the physics achievement test in a single session.They were provided with hardcopies of the questionnaires and optical mark sheets to shade their responses.

Results and Discussion
SPSS Version 19 (IBM Corp, 2010) was used to compute all descriptive statistics, Cronbach's α coefficients, and bivariate correlations.LISREL 8.8 (Jöreskog & Sörbom, 2006) was used to conduct all CFAs based on the maximum likelihood estimation method.Prior to the analysis, the scores of all negatively worded items were first reversed.Screening tests for conformity to underlying CFA assumptions were then conducted.These tests generally produced satisfactory results.Inspections of z-scores and Mahalanobis distances indicated no significant univariate or multivariate outliers at the .001level, and there was no evidence of multicollinearity between variables within the set.Skewness and kurtosis coefficients indicated no significant deviations from normality in the item distributions in terms of kurtosis, though moderate levels of skew across several items were observed.Given this, the PRELIS Normal Scores module in LISREL 8.8 was used to transform the scores prior to conducting the CFAs.Item descriptive statistics for the MSLQ motivation and learning strategies scales appear in Tables 6 and  7, respectively.

Confirmatory Factor Analyses
The internal structure of the MSLQ was first investigated by examining its factor structure.CFAs were performed separately for motivation and learning strategies scales, given that the two MSLQ sections are theoretically distinct.
For the motivation section, three competing nested models were tested, as presented in Table 8.Model M1 included all item statements from all motivation scales as one factor, given that these all assess facets of learning motivation.Model M2 tested a three-factor model, with items grouped into the three broad theoretical constructs (value, expectancy and affect) stipulated by Pintrich et al. (1991Pintrich et al. ( , 1993)) Another three models were tested for the MSLQ learning strategies section, as presented in Table 9. Model LS1 included the item statements for all the learning strategy subscales as one factor.Model LS2 tested a three-factor model, with items grouped into the three broad theoretical sub-constructs (cognitive, metacognitive and resource management) proposed by Pintrich et al. (1991Pintrich et al. ( , 1993)) Cudeck, 1993;Byrne, 1989;Schumacker & Lomax, 2004).The Goodness of Fit Index (GFI) was not used in this study, based on current recommendations within the field (Sharma, Mukherjee, Kumar, & Dillon, 2005).Differences between nested models within each of the MSLQ sections were evaluated using the chi square difference test (i.e., Δχ 2 ).
The fit indices obtained for each model of the MSLQ in this study are presented in Table 10.As indicated, for the motivation section, the one factor model did not fit the data well.Both the three-factor and the six-factor models met accepted cut-offs for the CFI and NNFI, though the SRMR for the three-factor fell above the recommended cut-off value.All three models differed significantly from one another based on the Δχ 2 test.Given these results, it was concluded that the six-factor model provided the best fit to the data.For the learning strategies scales, again, the one factor model did not fit the data well, though both the CFI and the NNFI fell only marginally below the accepted cut-values.Indeed, the χ 2 /df value for the one-factor learning strategies model fell within acceptable parameters.The three-factor model represented a significant improvement on the one-factor based on the Δχ 2 test, but this model still fell short of accepted cut-offs based on the SRMR and the NNFI.The nine-factor model was clearly the best-fitting, meeting accepted fit levels for the χ 2 /df, CFI, and NNFI, though the SRMR for this model was still somewhat high.Based on these results, the nine-factor was deemed to represent the best-fitting model.These results are comparable to those obtained in previous studies (e.g., Alkharusi et al., 2012;Pintrich et al., 1991;Karadenzi et al., 2008).The path diagrams (with standardized coefficients) for models M3 and LS3 are presented in Figures 1 and 2, respectively.Cronbach's alphas for each of the MSLQ scales are presented in Table 11.Kline (2000) suggested that an alpha coefficient of above .60represents an acceptable level of internal consistency, with alpha coefficients of above .70indicating a good level of consistency for low-stakes tests.As indicated, eight of the MSLQ scales had a good level of internal consistency, with six others achieving an acceptable level.Only one scale (Help Seeking) was observed not to meet the acceptable level of 0.6.These results are similar to those obtained by Pintrich et al. (1991).The poor result obtained for the Help Seeking scale is also consistent with previous studies.García and Pintrich (1995) attributed the low internal consistency of Help Seeking to the fact that it relates both to seeking assistance from peers and from teachers.García and Pintrich argued that students might be inclined to seek help from only one of the sources, which would contribute to the lower internal consistency of this scale.

Inter-Correlations between Subscales
The internal structure of the MSLQ was further assessed by examining correlations between the individual subscales.Results are presented in Tables 12 and 13, respectively.The pattern of correlations obtained aligned well with the theoretical basis of the instrument.Intrinsic Goal Orientation, Task Value, Control of Learning Beliefs and Self-efficacy were positively and highly correlated with one other, and all the self-regulated learning strategies factors were also positively correlated with one other.The one subscale that exhibited some unexpected characteristics was the Extrinsic Goal Orientation component.First, this subscale did not correlate negatively with Intrinsic Goal Orientation.This is not aligned with the views of many researchers (e.g., Harter, 1981), who propose the two constructs to be opposing ends of a single continuum.Furthermore, significant positive correlations were found between Extrinsic Goal Orientation and Task Value, Control of Learning Beliefs, and Self-efficacy for Learning and Performance.This might be deemed counter-intuitive, as many studies conducted in the West have found extrinsic rewards to undermine intrinsic motivation (e.g., Deci, 1971;Kruglanski, Friedman, & Zeevi, 1971;Lepper, Greene, & Nisbett, 1973).The findings of this study are, however, consistent with other studies involving East Asian students.For example, Lin et al. (2003) found that the highest performing Korean students exhibited high levels of intrinsic motivation as well as moderate levels of extrinsic motivation.Similar results were also found in a study by Kember, Wong and Leung (1999) on Hong Kong students.Such observations may reflect the Confucian Heritage cultures of these countries, which places great emphasis on hard work and academic success, and in which education is viewed as important not only for the individual, but also, for the family and society (e.g., Biggs, 1998;Salili, 1996).

Validity Evidence Based on Relationships with External Variables
Correlations between the MSLQ scales, R-LPQ-2F motive subscales, and physics achievement test scores are presented in Table 14.With the exception of extrinsic motivation, all obtained correlations aligned well with expectations.The pattern of correlations for Extrinsic Goal Orientation departed somewhat from previous results obtained in Western cultures.Extrinsic Goal Orientation correlated positively (though weakly) with Deep Motive and not with Surface Motive.Again, this may reflect the Confucian Heritage culture of Singapore.Specifically, it may be that even students who adopt extrinsic goal orientations will be motivated to use deep learning strategies, because these strategies are often needed for a high success level.(Biggs, 1998).Correlations between the MSLQ scale scores and physics achievement are shown in Table 16.As indicated, again, most of the motivation scales related as expected to achievement, with positive correlations obtained between achievement and Intrinsic Goal Orientation, Task Value, Control of Learning Beliefs, and Self-efficacy for Learning and Performance, and a negative correlation obtained between achievement and Test Anxiety.Extrinsic goal orientation, however, did not correlate significantly with test scores.Relationships between the MSLQ learning strategies scales and achievement (see Table 17) similarly aligned with expectations, though Rehearsal, Organisation, and Help Seeking were not significantly correlated with achievement test scores.Overall, however, these results suggest that the MSLQ scores related to achievement in alignment with the theoretical basis of the instrument.

Conclusion
Results of this study indicate that the internal structure of the MSLQ in Singapore high school students is similar to the original factor structure proposed by Pintrich et al. (1991).In this study, a structure comprising six motivation scales and nine learning strategies scales fit the data well.The internal consistencies of the subscales were also generally within acceptable ranges.While the Help Seeking subscale did exhibit a relatively low  coefficient, this aligns with the results obtained in the original validation of the scale.The latter result may signal the need for further refinements of this particular scale.Results of the inter-factor correlations within the MSLQ and correlations with external variables also generally supported the construct validity of the instrument.Overall, these findings suggest that the MSLQ is an appropriate measure of students' learning motivations and strategies in the Singapore secondary context.Future research is needed to determine whether the instrument is suitable for use in other grade levels within this system.

Figure 1 .
Figure 1.Original six-factor model of the MSLQ motivation subscales

Table 1 .
Structure and sample item statements of the MSLQ

Table 3 .
Expected correlations between MSLQ learning strategy scales and R-LPQ-2F strategy subscales

Table 4 .
Expected correlations between MSLQ motivation scales and physics achievement

Table 5 .
Expected correlations between MSLQ learning strategy scales and physics achievement Approval to conduct the research was first obtained from the Human Research Ethics Committee of the University of Western Australia.All procedures used within the study were conducted in compliance with the National Health and Medical Research Council's (2007) Australian Code for the Responsible Conduct of Research.

Table 7 .
. Model M3 tested a six-factor model based on the scales proposed by Pintrich et al. (Intrinsic Goal Orientation; Extrinsic Goal Orientation; Task Value; Control of Learning Beliefs; Self-Efficacy for Learning and Performance; Test Anxiety).Item descriptive statistics for the MSLQ learning strategies scale

Table 8 .
Models tested for the motivation scales of the MSLQ Two absolute fit indices (the Standardized Root Mean Square Residual, or SRMR, and the relative chi-square value, or χ 2 /df) and two relative fit indices (the Comparative Fit Index, or CFI, and the Non-Normed Fit Index, NNFI) were used to assess the fit of each model tested.Accepted cut-offs suggest that in CFAs, good model fit is indicated by a χ 2 /df < 5 and SRMR < 0.08, with values greater than 0.90 for the CFI and NNFI (e.g.,Browne &

Table 10 .
Fit indices alternative models of motivation and learning strategy scales

Table 11 .
Internal consistency of the MSLQ subscales

Table 12 .
Inter-correlations of the MSLQ motivation scales

Table 13 .
Inter-correlations of the MSLQ learning strategies scales

Table 14 .
Correlations of the MSLQ motivation scales with R-LPQ-2F

Table 15
presents correlations between MSLQ learning strategies scales, R-LPQ-2F strategy subscales, and achievement.As expected, most of the MSLQ learning strategies scales were positively correlated with Deep Strategy, and negatively or not significantly correlated with Surface Strategy.Help Seeking was an exception, but there were no theoretical grounds for this subscale to exhibit specific correlation patterns with deep or surface learning strategies.Interestingly, it was found that Rehearsal, which is a basic cognitive strategy, was positively correlated both with surface and with deep learning strategies.While its positive correlation with deep strategies is not aligned with previous studies conducted in Western cultures, this is again consistent with previous studies involving students from Confucian Heritage cultures.Strategies including repetition and memorizing are commonly used by Asian students who engage in deep learning, as a precursor to other strategies

Table 15 .
Correlations of the MSLQ learning strategy scales with R-LPQ-2F learning scales and physics achievement test

Table 16 .
Correlations of the MSLQ motivation scales with R-LPQ-2F motive scales and physics achievement test

Table 17 .
Correlations of the MSLQ learning strategy factors with R-LPQ-2F learning factors and physics achievement test