VALIDATION OF TEACHING AND LEARNING GUIDING PRINCIPLES INSTRUMENT FOR MALAYSIAN HIGHER LEARNING INSTITUTIONS

Purpose – This study was aimed at establishing, through the validation of the Teaching and Learning Guiding Principles Instrument (TLGPI), the validity and reliability of the underlying factor structure of the Teaching and Learning Guiding Principles (TLGP) generated by a previous study. Method – A survey method was used to collect data through the 67-item TLGPI. The TLGPI was hypothesised to measure the six constructs of the TLGP. The TLGPI was administered online to all academicians from 20 public universities in Malaysia. In total, 284 responses were received from a population of 31,608 academicians. The collected data were subjected to a Confirmatory Factor Analysis to test for construct reliability, model fit, and convergent validity of each contruct’s measurement model. The CFA was also used to test the model fit and the discriminant validity of the pooled measurement model. Findings – The CFA supported a finalised measurement model of six constructs for the TLGP. The TLGP was found to have reliable and valid constructs that represented the aspects of teaching and learning that are important in Malaysian higher learning institutions. Significance – This pioneer study is vital in the Malaysian higher ht tp :// m jli .u um .e du .m y 126 Malaysian Journal of Learning and Instruction: Vol. 13 No. 2 (2016): 125-146 education context since the development of teaching and student learning can benefit from a set of empirically validated teaching and learning principles. The TLGP are statements on the scholarship of teaching and learning and can be a reference guide to good practice that can help set the pathway to a high-quality learning environment for students in the Malaysian higher learning institutions and the wider Asian context.


INTRODUCTION
The development of teaching and student learning in higher education is a key issue for universities worldwide (Olsson & Roxa, 2012).Now more than ever, many nations around the world are actively making a serious effort to reform and transform their education systems.The latest effort by the Malaysian Ministry of Education is documented in the Malaysia Education Development Plan (Higher Education) 2015-2025, launched in April 2015. .Ten key issues are emphasised in the plan, and the first issue, which is closely related to our overall research, is producing graduates who are holistic, have entrepreneur characteristics and are balanced.
Therefore, it is imperative that all higher education institutions in Malaysia strive to improve their graduates' quality across intellectual, spiritual, emotional, and physical dimensions to enable them to thrive in today's complex global economy and contribute to the betterment of society.One of the strategies is to institutionalise a holistic and integrated curriculum that includes the necessary quality design, delivery and assessment aspects.The central player in ensuring that these aspirations become a reality is the educator, since quality educators are the definitive determinants of student achievement.Our related study (Nurulhuda & Azwani, 2014) has shown that more than 80% of the educators in the sample maintained the traditional mode of delivery, the i.e., by using standard lecturing where they explained the materials while students passively listened and perhaps took notes.A study by Siew et al. (2015) showed that students in one of the Malaysian universities relied on their educators for information and were not inclined to explore on their own.A more learner-centred approach to teaching and learning where students have to be actively engaged during a lesson would be challenging to implement, given this prevalent scenario.
The development of teaching and student learning can benefit from a set of teaching and learning principles.Teaching and learning principles are statements on the scholarship of teaching and learning and a reference guide to good practice.These principles represent the shared view within an institution of the processes and conditions that contribute to a high-quality teaching and learning process (University of Melbourne, 2007).Elements of the principles should be embedded in the fundamental philosophy adopted by a particular institution and reflect the institutional commitment in providing a high-quality teaching and learning ecosystem.Together, these principles should reflect the conditions under which student learning can thrive and thus, have a direct bearing on the quality of student intellectual development.
For our research, the TLGP will be used to develop a framework for teaching and learning.We hope that the TLGP can be used by educators from different backgrounds and disciplines to work together to plan, develop, and provide coherent interdisciplinary learning experiences for students.Thus, in general, teaching and learning principles can be a guide to the maintenance and enhancement of teaching and learning standards.The suggestions of good practice offered by the guidelines provide benchmarks to which the institution is committed within the parameters of available institutional resources.The ultimate objective of the institution's teaching and learning programs should be to prepare graduates with distinctive attributes that enable them to contribute to the everchanging global context in a meaningful and positive way, in line with the aspirations stated in the Malaysia education blueprint.

REVIEW OF LITERATURE
There are very few empirical studies that develop teaching and learning principles.We have not found a single study that develops and generates teaching and learning principles such as the one we are developing.One empirical study that is somewhat related, but focuses only on the teaching and learning of web-based distance education was by Kanuka (2002).The outcome was the development of a model with nine principles that facilitate higher levels of learning in Web-based distance education.All nine principles resonated well with our principles.For example, principle 1 (learning that involved complex abstracted phenomena), principle 2 (learning that included diverse and/or multiple perspectives), principle 3 (learning that included phenomena that had personal relevance), principle 4 (learning that included diverse ways of knowing), principle 5 (learning that included an assessment process that was personally meaningful), principle 6 (learning that required learners to assume greater responsibility), principle 7 (learning that required learners to build meaning), principle 8 (learning that required learners to understand their worldview) and principle 9 (learning that required learners to provide evidence of new understanding) were all aligned with our principle 1 (teaching and learning that fostered an atmosphere of intellectual excitement), principle 4 (constructive alignment between an evolving knowledge base, student learning outcomes, learning experiences, actual practice, and assessment), principle 5 (international and culturally diverse learning environment) and principle 6 (climate of inquiry and critical reflection).In validating these principles, the study used expert validation through a two-stage focus group examination of the generated provisional principles.For our research, after conducting the expert validation process, we validated our principles further using the rigour of CFA statistical analysis.Korthagen, Loughran and Rusell (2006) sought to develop principles for teacher education programs and practices unlike our narrower concept of teaching and learning as the delivery of the curriculum.In generating their principles, the researchers analysed three education programmes from different continents and used a meta-analysis of different studies on the programmes.Analyses done were primarily qualitative in nature, for example through the use of concepts such as naturalistic generalisability, catalytic validity, dialogic validity and pattern matching.The outcome was seven principles of learning about teaching.Since the seven principles were focused on change in teacher education, they were more encompassing in the way they were stated.For example, principle 1 stated that learning about teaching involved continuously conflicting and competing demands.As teacher education could not adequately prepare teachers for their entire careers, teacher education programmes inevitably needed to respond to a range of conflicting and competing demands (Korthagen et al., 2006).However, our principles were more focused on the good practices within the classroom itself.
An institution or center usually developed teaching and learning principles based on inputs from a group of scholars or experts within and outside the institution.Thus, the methodology of generating these principles could not be gleaned from the literature.Teaching and learning principles developed by these institutions usually referred to the overall teaching and learning environment, and were not limited to the delivery of the curriculum.However, our research was part of a bigger research project aimed at developing a model of teacher education by focusing on five components of teacher education, namely curriculum, teaching and learning, assessment, teacher leadership, and clinical experience and teacher induction.Guiding principles for each of the components will help inform the development of a teacher education model.
Several examples of teaching and learning principles included the ones developed by the Eberly Centre and Queensland State Schools.The principle of effective teaching developed by the Eberly Centre (Carnegie Mellon-Eberly Center, 2008) comprised seven themes, namely analysing student needs, constructively aligning learning objectives, assessments and instructional activities, articulating explicit expectations, prioritising certain knowledge and skills, recognising and overcoming their expert blind spot, adopting appropriate teaching roles, and progressively refining courses based on reflection and feedback.
Queensland State Schools formulated five guiding principles for the development and implementation of quality learning programs through promoting effective learning and teaching (Education Queensland, 2012).In essence, the principles stressed the need to understand the learner and the learning process, provide a supportive and challenging environment, establish worthwhile learning partnerships, and shape and respond to a variety of social and cultural contexts.These principles were based on the assumptions, among others, that the principle of effective learning and teaching provided the basis for the ongoing improvement of learning and teaching practices.It is clear that both the Eberly Centre and Queensland State Schools have adopted principles that encompassed not only the delivery of curriculum, but also the whole teaching and learning ecosystem.
Another illustrative example is from the Teaching and Learning Research Programme (TLRP) in the UK that has developed evidenceinformed pedagogic principles, the product of an iterative process of consultation and debate between researchers, practitioners, policymakers and the TLRP Directors' Team (Teaching and Learning Research Programme, 2007).They generated 10 principles that included themes such as equipping learners for life, engaging with valued forms of knowledge, recognising the importance of prior knowledge and experience and scaffolding learning, promoting active engagement and the need for constructive alignment between assessment and learning, fostering both individual and social processes and outcomes, and recognising the significance of informal learning.They also demanded a consistent policy framework with support for learning as the primary focus.However, there was no accessible documentation on the processes in developing these principles.

BACKGROUND OF THE RESEARCH
The purpose of this article is to present the processes and analyses involved in further validating the Teaching and Learning Guiding Principles (TLGP) in the context of higher learning institutions in Malaysia by Adnan, Masuwai, Tajudin, & Rahman (2015).The previous stage of the research resulted in six key themes, later labeled as constructs, related to principles of teaching and learning.They were: (a) nurture good values, (b) intellectual excitement, (c) quality learning spaces, technology and resources, (d) constructive alignment, (e) international and cultural diversity, and (f) climate of inquiry and critical reflection.The process involved several steps of document analysis, content analysis and gap analysis.Data were gathered from three sources, namely the literature on teaching and learning theories, policies and practices, teaching and learning guiding principles from selected institutions, and 10 key informants.
Each of the constructs was subsequently expanded into a sentence or a phrase to form a statement of principle, as shown in Table 1.Constructs were defined as the essential elements that constituted each principle.Corresponding to each of the six guiding principles, 10, 16, 12, 7, 10, and 12 items were generated respectively, giving a total of 67 items.The items were put together in the form of a questionnaire and named the Teaching and Learning Guiding Principles Instrument (TLGPI).We used a 5-point Likert Scale; from 1: 'Not important at all' to 5: 'Extremely important', to assess the importance of each of the items in the context of teaching and learning in higher education.The responses were assigned values of one to five, respectively.

Six Teaching and Learning Guiding Principles (TLGP)
No.
Teaching and Learning Guiding Principles 1 Nurture good values, attitudes and behaviours 2 Foster an atmosphere of intellectual excitement in students 3 Quality learning spaces, resources and technologies 4 Constructive alignment between an evolving knowledge base, student learning outcomes, learning experiences, actual practice, and assessment 5 International and culturally diverse learning environment 6 Climate of inquiry and critical reflection In this study, further validation of the constructs underlying the TLGP was conducted using Confirmatory Factor Analysis (CFA) through a validation of the TLGPI.More specifically, our research objectives were: a. to establish, using CFA, the construct reliability, model fit and convergent validity of the constructs underlying the TLGP; b.
to establish, using CFA, the model fit and discriminant validity of the pooled measurement model of the TLGP.

Sample
We administered the TLGPI online through a link in emails that were sent to all 20 public institutions of higher learning in Malaysia.
Our request to forward the link to all academicians in the public universities was made with the assistance of each institution's public relations officer.Altogether a total of 284 responses were received from a population of 31,608 academicians, which gives a response rate of 0.90%.The population size was estimated from the number of academicians as posted on the official website of the Ministry of Higher Education for 2014.Some researchers (e.g.Nevitt & Hancock, 2001) recommended a minimum sample size of 100, while others recommended at least 200 for reasons of accuracy (Kenny, 2012;Kline, 2011).Accordingly, a sample of 284 for this study was considered adequate.

Instrument
As stated earlier, the instrument used to collect further data was the TLGPI, the same instrument used in the previous stage of the research (Adnan et al., 2015).In validating the factor structure underlying the TLGP, we were in effect validating the factor structure of the TLGPI.

Data Analysis
Data analysis involved an examination of construct reliability and included internal reliability and composite reliability.In addition, in order to establish the underlying structure of the measurement used, the data were subjected to a confirmatory factor analysis using AMOS 22.0.The analysis served to validate that the model developed by the researchers fit the data collected.The comparison between the previous study about the TLGP and this study is as shown in Table 2.

Comparision of Previous and Present Study of Teaching and Learning Guiding Principles (TLGP)
Previous study (Adnan et al., 2015) This study Reason The EFA study informed the mean, standard deviation and internal consistency of the constructs.
The overall value of Cronbach's alpha was 0.977.CFA was employed to test whether the measures of the construct were consistent with the researchers' understanding of the nature of the constructs (Awang, 2014b).

EFA
CFA was used to confirm that the indicators sort themselves into factors corresponding to how the researcher had linked the indicators to the latent variables.
EFA was used as an exploratory first step in the development of a measure, and then CFA was used to confirm the factor structure identified in EFA (Harrington. 2009).
EFA was more data driven while CFA was more theory-driven (Harrington. 2009).

Evidence of Unidimensionality
Unidimensionality refers to how well the various items or indicators measure the same latent construct.Unidimensionality assessment should be made before the assessment of reliability and validity (Awang, 2014a).The assessment of unidimensionality can be determined by an item's factor loading.Hair et al. (2006) suggested that an item was significant if its factor loading was greater than 0.50.Five items were deleted because they produced low factor loadings.

Evidence of Construct Reliability
The assessment of reliability for a measurement model can be determined by calculating the internal reliability (α > 0.7) and composite reliability (CR ≥ 0.6) values.Based on the reliability analysis, the internal reliability showed an overall Cronbach alpha value of 0.976 which exceeded the acceptable value of 0.70 (Hair et al., 2010).Furthermore, the Cronbach alpha value for each construct ranged from 0.877 to 0.930.For composite reliability, the overall value was 0.868 and the individual values ranged from 0.841 to 0.911, well within the range suggested by Robinson et al. (1991) and DeVellis, Lewis, and Sterba (2003).The high reliability for each of the six constructs seemed to indicate that the items amalgamated well within each construct.Table 3 shows the details of the reliability values for all of the constructs.

Evidence of Model Fit for Individual Constructs
Model fit for this stage of the analysis referred to how well our hypothesised model (in this case, the model of the factor structure for each construct) accounted for the correlations between variables in the dataset.A good fit showed that the model was plausible.To test for model fit, we assessed for fit indices.Among the available fit indices were relative chi-square (CMIN/DF), Tucker-Lewis Index (TLI), Comparative Fit Index (CFI), Incremental Fit Index (IFI), Normed Fit Index (NFI), and Root Mean Square of Error Approximation (RMSEA).For a sample size of more than 250, but less than 1000, and with the number of indicators of more than 30, the criteria of the fit indices suggested are as summarised in Table 4.

Table 4
The Criteria for Fit Indices  Hair et al. (2010) and Holmes-Smith (2006) recommended using at least one fit index from each category of the model fit.Hair et al. (2010) also suggested three to four fit indices for establishing model fit.For this study, we chose the RMSEA for the absolute fit category; the CFI, IFI, NFI for the incremental fit category; and the CMIN/df for the parsimonious fit category as these indices achieved the recommended values for all of the constructs.Table 5 shows the assessment of fitness for the model structure of each construct.Based on the suggestion by Hair et al. (2010), all constructs achieved the recommended value for at least three of the fit indices, suggesting that the model fits the data.Model fit was obtained after several items were deleted while others correlated.

Evidence of Convergent Validity
According to Kline (2005), convergent validity refers to a set of indicators that is presumed to measure a construct.Brown (2006), however, defined convergent validity as the internal consistency of a set of items or indicators.These definitions are in line with the concept of unidimensionality.It represents the strength of relationships between items that are predicted to represent a single latent construct.To test for convergent validity, we determined the value of the factor loadings and average variance extracted (AVE).
The values of factor loadings were reported earlier in the subsection on evidence for unidimensionality.The AVE is a more conservative test of convergent validity (Malhotra & Dash, 2011).It measures the amount of variance captured by the construct in relation to the amount of variance attributable to measurement error.Convergent validity is judged to be adequate when average variance extracted equals or exceeds 0.50 (i.e., when the variance captured by the construct exceeds the variance due to measurement error).All of the constructs had an AVE of above 0.5 as shown in Table 5.

Evidence of Model Fit for the Pooled Measurement Model of the TLGP
The measurement model for all of the constructs (pooled measurement model) involved in the study was then assessed.Using this method the correlations between constructs were computed simultaneously (Awang, 2014a).Table 6 shows the fit indices of the proposed pooled measurement model.The initial finding shows that the chosen fit index tests such as the TLI, CFI, IFI and NFI were still less than 0.90 as set by the requirement.However, the values of the CMIN/DF and RMSEA achieved the minimum cut-off point.

Table 6
The fit indices for the proposed pooled measurement model  and NV59 (16.159).All of the items were deleted because of their high MI values.Creating correlations between IE10 and IE11; IE10 and IE13; IE15 and IE16; CI51 and CI56; CI51 and CI52; and CI50 and IC44 had increased the required value as is shown in Table 7, and the revised pooled measurement model is shown in Figure 1.

Table 7
The fit indices for the revised pooled measurement model The relative Chi-square was at an acceptable value of 2.046 while the other fit statistics were more than the cut-off value of 0.90.The RMSEA value of 0.060 was much better than the suggested value of 0.08.Only the value of the NFI (0.847) was still less than 0.90 as set by the requirement, but it was still acceptable (Hair et al., 2010;Holmes-Smith, Cote, & Cunningham, 2006).In conclusion, the pooled measurement model of TLGP fit the collected data.

Evidence of discriminant validity
Discriminant validity measures the extent to which constructs are different.Discriminant validity is present when the correlation between each pair of the latent construct is less than 0.90 (Hair et al., 2010).It indicates that each construct shared more variance with its items than it does with other constructs.Table 8 shows the correlation matrix for the constructs where the correlation coefficients values between constructs were all less than 0.90.Therefore, the discriminant validity of all six constructs in this study was achieved.
In conclusion, all six constructs in the pooled measurement model were deemed significantly different.

Evidence of content validity after several items were deleted
According to Haynes et al. (1995), content validity is a vital component in construct validity.It provides the evidence about the degree to which the elements of the assessment instrument are relevant to and representative of the targeted construct.The previous study had reported the process of content validation for the original 67 items.For the remaining items in this study, nine experts were identified and invited to review the instrument for content validity.The Content Validity Index (CVI) was used to indicate the degree of agreement among the experts (Lynn, 1986).A favourable rating among the experts yielded a CVI of 0.989 or 98.9%, denoting a high level of agreement (Lawshe, 1975) as is summarised in Table 9.  6-7, 9-14, 18, 20-21, 28, 32-34, 36, 38-40, 44, 50-52, 56-58

DISCUSSION
This study was part of a larger study aimed at developing a framework for teaching and learning in a higher education environment.The purpose of this study was to further validate the guiding principles that were developed from the previous stage of the research.The constructs that underlay the principles would form the core elements of the framework.Teaching and learning principles could be used as a guide to facilitate higher levels of teaching and learning.In the context of teacher education institutions, the AERA panel on Research and Teacher Education (Cochran-Smith & Zeichner, 2005 in Korthagen et al., 2006) made the conclusion that there was no clear evidence that certain approaches in teacher education might be more effective than others.Furthermore, they questioned the effectiveness of teacher education on student teachers' performance.However, Brouwer and Korthagen (2005) argued that this conclusion was made based on a comparison of the general instructional strategies used in a program.In other words, they were compared at a high level of abstraction.They claimed that more specific principles guiding the practices within a program might lead to clear and desired outcomes in the graduates of such a program.Since the development of our principles are based on a narrower concept of teaching and learning, we believe that our principles are specific enough to guide educators and students towards making a significant improvement in how teaching and learning are practiced.However, we are well aware of the fact that any desired change in the practices of teaching and learning is an in-depth process that may take many years to become effective.It also requires a corresponding change in thinking about educating learners, and the enormous implications for educators, which in turn, will involve an attitudinal shift (Korthagen et al., 2006).

CONCLUSION AND IMPLICATIONS
In further validating the TLGP, the analysis has yielded six reliable and valid factors of the TLGP, namely Intellectual Excitement, Quality Learning, Constructive Alignment, International Cultural, Inquiry Reflection and Good Values.After undergoing the iterative process of validation, a total of 37 items were deleted, and three covariances were created as suggested by the modification index.A summary of the validation process is as shown in Table 10.The goodness of fit indices of the CFA indicated a good fit of the collected data and the model.Hence, while we have retained all six constructs from the previous research, the number of items or indicators of the constructs have been reduced to 30.However, we have strengthened the psychometric properties of the TLGPI by showing the evidence for construct reliability, model goodness of fit, convergent validity and discriminant validity.We had hypothesised that the TLGPI measured the constructs of the TLGP, thus by strengthening the TLGPI, we had correspondingly strengthened the measurement model of the TLGP.While the main purpose of the research was to validate the model structure of the factors underlying the TLGP, we were also able to validate the instrument (i.e., the TLGPI) that we used to collect data on aspects that were important in guiding teaching and learning in higher learning institutions.We therefore, conclude that our TLGP has reliable and valid constructs that represent the aspects of teaching and learning that are important for Malaysian Higher Learning Institutions.
The relevant constructs of teaching and learning can help quantify aspects of teaching and learning from the perspective of educators in higher learning institutions.And with this validated instrument, future research in developing teaching and learning principles will benefit from the enhanced psychometric aspect.The present study can be extended to different settings and sample in order to achieve a better measurement model.Although our number of participants exceeded the minimum number required for running the CFA, it only constituted 0.90% of the total population.It is hoped that by validating the teaching and learning principles, a broader spectrum of related issues will be addressed in the future in order to better guide teaching and learning practices.
This study is vital in the context of Malaysian higher learning institutions and can potentially be useful in the wider Asian context since every institution needs to have a set of principles that can guide the teaching and learning practices so as to ensure a high-quality learning environment is provided for students.However, it remains arguable whether these principles are functional and practical and, as such, it is still in need of further research.Specifically, while few would argue against the view that educators need to use theoretical principles of learning to guide instruction, it is often not easy to apply the theoretical principles in actual practice.Thus, further research is needed to determine if the model achieves this aim through an investigation of perceived usefulness by practitioners, researchers, and learners.Nevertheless, the aspirations stated in the Malaysia education blueprint for higher education are in line with the principles developed in this study.

Figure 1 .
Figure 1.The revised pooled measurement model of the TLGP

Table 3
Reliability Values of Constructs of the Teaching and Learning Principles

Table 5
Assessment of fit indices for the model structure of each construct

Table 5
Convergent Validity tests results

Table 8
The Discriminant Validity Index Summary Based on the Correlation Coefficient Values

Table 10
Summary of the Validation Process and Number of Items of the TLGPI