Examining teachers’ technological pedagogical and content knowledge in the era of cloud pedagogy

With the ongoing innovation of instructional technologies there has been an emerging call to examine what types of knowledge teachers require to survive in the era of cloud pedagogy. In response to this call we proposed a research model – TLPACK – which is based on technological pedagogical content knowledge (TPACK), information communication technologies technological pedagogical content knowledge (ICT-TPCK), and education technology, pedagogy and didactics, academic subject-matter discipline, educational psychology and educational sociology knowledge (TPACK-XL), to explore the types of knowledge that teachers at various levels – from kindergarten to post-secondary level – should equip themselves with in detail. TLPACK consists of five constructs (technology knowledge, learner knowledge, pedagogy knowledge, academic discipline, content knowledge, and context knowledge) but in total the TLPACK scale comprises 39 items. All items were converged based on the viewpoints of five experts from academia and practice following six rounds of the Delphi method, and the finalised version was prepared for reliability and validity examination. Proportional stratified sampling was adopted to conduct a questionnaire survey among teachers from kindergarten to post-secondary levels in Taiwan (n = 301). Rigorous statistical analyses were undertaken to examine the reliability and validity of this new model. Based on the results of statistical analyses, including item analysis, exploratory factor analysis, and confirmatory factor analysis, it is reasonable to state that the proposed TLPACK scale has good reliability and validity for practical use. The conclusion and limitations of this study were drawn based on the extracted results, and suggestions for future study are reported at the end of this report.


Introduction
The expeditious development of information communication technologies (ICTs) has significantly changed the current economic structure worldwide and therefore alters the ways people undertake business transactions, and has also led to pedagogical transformation (Nkula & Krauss, 2014;Ramoroka, Tsheola & Sebola, 2017) to equip students with the needed skills in the industry (Dede, 2009;Koh, Chai, Benjamin & Hong, 2015;Leendertz, Blignaut, Nieuwoudt, Els & Ellis, 2013). Teaching is a complex activity that involves many factors and types of knowledge (Gill, 2015;Mishra & Koehler, 2006). In recent years new technologies have reshaped teachers' mindsets about teaching and altered learners' learning behaviour (Kinshuk, Chen, Cheng & Chew, 2016). Moreover, it has been noted that a new mindset on teaching and learning is required because current students are considered the "Net Generation" (Oblinger & Oblinger, 2005); their aptitudes, attitudes, expectations, and learning styles are accordingly different from those of traditional learners (Price, 2015), and in this regard, Hwang (2014:11) suggests that "new learning modes will raise new pedagogic issues." The rapid progress of Web 2.0 technologies has made virtual learning and teaching a trendy way of acquiring and delivering knowledge, and cloud pedagogy is just one of many novel breakthroughs in education practice (Wang, Chen & Khan, 2014). Barak (2017) coined the term "cloud pedagogy" as a new paradigm of teaching which encourages teachers to undertake knowledge delivery ubiquitously through cloud-based applications, which is used to describe a framework of instruction in which to practice constructivism through socialisation. Even educational practitioners in remote rural regions such as some areas in South Africa are aware of the affordance and advantages of technology in teaching and learning, and thus are motivated to acquire pertinent knowledge to succeed in this innovative pedagogy (Conger, Krauss & Simuja, 2017;Leendertz et al., 2013). A study by Blanchard, LePrevost, Tolin and Gutierrez (2016) indicates that teachers who received related training on the use of technologies may benefit by increasing their teaching effectiveness, because students would have stronger motivation to engage in the instructional activities, which would lead to better learning outcomes. Hence, teachers are expected to include various types of technologies to effectively deliver the target contents and create more learning opportunities for learners (Angeli & Valanides, 2009).
Bearing this in mind, Mishra and Koehler (2006) consider that knowledge about integrating technology into teaching should be included as a part of teachers' basic pedagogical knowledge. Shulman (1987) proposes seven basic aspects of the competences with which a teacher should be equipped, namely content knowledge, curriculum knowledge, pedagogical content knowledge, knowledge of learners and their characteristics, knowledge of educational contexts, knowledge of educational ends, purposes, and values, and their philosophical and historical grounds. He further proposed the PCK model. Based on Schulman's PCK model, Mishra and Koehler (2006) took into account the expeditious development of technologies in education and proposed including knowledge of technology as part of teachers' core knowledge. They developed the TPACK model, which has been widely used in pertinent research (Aydin, Evren, Atakan, Sen, Yilmaz, Pirgon, Yeşilyurt, Akıllıoğlu & Ebren, 2016;Ellis, Dare & Roehrig, 2016;Hilton, 2016). Subsequently, Angeli and Valanides (2009) note the importance of teachers' knowledge about using ICTs such as mobile phones and tablets in their instruction, suggesting the ICT-TPCK model, which also considers teachers' knowledge about learners and context (referred to as learners knowledge and context knowledge in the model). In 2012 Saad, Barbar and Abourjeili extended the applicability of the ICT-TPCK model and suggested illuminating the salient issue of its relatedness to teacher education courses. The TPACK-XL model was accordingly developed.
However, it has been shown that TPACK (Mishra & Koehler, 2006) and other models derived from TPACK, such as the ICT-TPCK (Angeli & Valanides, 2009) and TPACK-XL (Saad et al., 2012), cannot provide a full picture of good teaching if the above points of knowledge (technology, pedagogy and content knowledge) are discussed separately and therefore calls for a holistic view on teachers' general and professional competencies (Mishra & Koehler, 2006). Teachers need to adjust their teaching based on the given situation (Ben Hamida, Maaloul & Ben Hamida, 2016), therefore, it is necessary to develop appropriate tools for teachers of various subjects to validate their teaching and allow them to explore various elements of instruction and more easily adjust their teaching (Chang & Ting, 2016). It is hoped that the development of a theoretical conceptual framework will help to conceptualize and structure theories and transform teachers' teaching pedagogy and practice (Mishra & Koehler, 2006). Indeed, the education sector recognises the worth of conceptual frameworks, particularly when teachers evaluate their teaching (Archambault & Barnett, 2010). Thus, the major purpose of this research was to develop a scale of constructs and indicators to indicate teachers' performance when practicing pedagogy which integrates technologies in class.
The Basis of the TLPACK Model: TPACK, ICT-TPCK, TPACK-XL The TLPACK model developed in this study is based on three previous models: TPACK, ICT-TPCK, and TPACK-XL. Mishra and Koehler (2006) postulated the TPACK model, which is made up of three major constructs, technological knowledge, pedagogical knowledge, and content knowledge, as a way to describe teachers' knowledge structure. The interaction between the three major constructs result in seven different aspects namely, technological knowledge (TK), pedagogical knowledge (PK), content knowledge (CK), technological pedagogical knowledge (TPK), technological content knowledge (TCK), pedagogical content knowledge (PCK), and TPACK. The TPACK model is illustrated in Figure 1 below.   Grunbaum's (1975) Venn diagram to explore the relationship between the five con-structs of education technology, pedagogy and didactics, academic subject-matter discipline, educational psychology, and educational sociology. The interaction of these five constructs in turn leads to 31 variables. The concept of TPACK-XL is depicted by the Figure 3 below. Generally speaking, TPACK, ICT-TPCK, and TPACK-XL are all derived from Shulman's (1986Shulman's ( , 1987 seven genres of knowledge. However, when Shulman (1987) posited his model, technology was not in widespread use in the educational context and for this reason, it did not take into account the use of, for example, cloud technology in education (Mishra & Koehler, 2006). Nevertheless, with the rapid development of various types of technologies in instruction such as Web 2.0 and mobile devices, technology knowledge cannot and should not be excluded from the repertoire of basic knowledge a teacher is expected to acquire (Mishra & Koehler, 2006). One matter that needs to be raised is that this basic knowledge is tightly interconnected and thus the structural relationship between the variables discussed above should be the focal point of further exploration (Angeli & Valanides, 2009;Mishra & Koehler, 2006;Saad et al., 2012).
Thus, the major reason for our attempt to develop the TLPACK model is because the other models mentioned have greater or lesser downsides in terms of describing what knowledge teachers are expected to have. Specifically, TPACK does not consider other teaching factors apart from content knowledge, pedagogical knowledge, and technological knowledge such as learners' knowledge of the target contents (Adam, 2017;Angeli & Valanides, 2009;Peng & Daud, 2016), and its constructs cannot be distinguished. This leads to a reduction in the predictive force or effect of developing new knowledge (Archambault & Barnett, 2010). Secondly, ICT-TPCK does not explain the interconnection between constructs (Saad et al., 2012), and it is difficult to assess easily and clearly (Albion, Jamieson-Proctor & Finger, 2010). Finally, TPACK-XL has been proposed as a suggested model to assess the pre-service teacher's curriculum and its relevance. However, the actual application of TPACK-XL cannot be determined as there is no support from other related research. However, TPACK-XL redraws the constructs of ICT-TPCK, establishing the interaction between them. Therefore, we decided to follow the framework of TPACK-XL. Moreover, all constructs of TLPACK are named in accordance with those used in TPACK, ICT-TPCK, TPACK-XL, or their connotations. The proposed research model is illustrated in Figure 4.

Figure 4
The TLPACK model As shown in Figure 4, TLPACK is composed of five constructs, namely, technology knowledge, learner knowledge, pedagogy knowledge, academic discipline content knowledge, and context knowledge, and the respective interactions between any of the abovementioned five constructs.

Research Design
We adopted the Delphi method and a survey research design to develop the constructs of the proposed research model. Indicators of constructs were proposed after reviewing the relevant literature. A panel of five experts (two of them professors of educational technology from two national universities in Taiwan, and the other three senior teachers at the secondary level who had at least ten years of experience using technologies in teaching) was invited to use the Delphi method to reframe the indicators as question items. All the experts had substantial teaching experience in various subjects including languages, mathematics, computer science, and special education. In the Delphi process, all experts remained anonymous; their only communication on the research was through email with the research team only. In other words, they never discussed the research project among themselves, either in public or in private.
After six rounds of the Delphi method the experts' opinions were converged and the final draft of question items was distributed to teachers in Taiwan. As the data had been collected, statistical analyses such as item analysis, exploratory factor analysis, internal consistency reliability as well as confirmatory factor analysis were applied to examine the reliability and validity of the questionnaire items.

The Delphi Method
The Delphi method used in this research comprised six rounds of iterations to reach convergence and the detail of each round's convergence is presented in Table 1 below. After the Delphi method had been completed, the first draft of TLPACK was developed with 78 question items based on five constructs -13 question items for technology knowledge, 19 question items for learner knowledge, 14 question items for pedagogy knowledge, 12 question items for academic discipline content knowledge, and 20 question items for context knowledge.

Questionnaire Survey
The finalised questionnaire was distributed through the internet as a Google survey, and 353 copies with successful responses to the TLPACK question items on a five-point Likert-style questionnaire (5 = strongly agree, 1 = strongly disagree) were re-trieved. The collected data was first examined using the skewness and kurtosis test, and only items with a skewness of less than .7 and a kurtosis of less than 2.58 were retained, as suggested by Tabachnick and Fidell (2007). Following data cleaning, 330 (n = 330) valid questionnaires remained among these responses, and we selected 301 samples through stratified sampling based on the level at which the participants were teaching. The 301 samples (n = 301) were tested for further reliability and validity analyses. Selected samples covered teachers of various levels from kindergarten to the post-secondary level in Taiwan. Information about the samples is reported in Table 2.  (Kaiser, 1974). We adopted the Method of Maximum Likelihood and Promax rotation to conduct EFA, and five factors were set to be extracted. In terms of factor loading, in order to increase the reliability and validity of the questionnaire, we partialed out those factors with a loading of lower than .5 or those that were cross-loaded (Hair, Anderson & Tatham, 1987;Lai, Hwang, Liang & Tsai, 2016). EFA extracted 39 items including 10 question items for technology knowledge, seven question items for learner knowledge, five question items for pedagogy knowledge, 12 question items for academic discipline content knowledge, and five question items for context knowledge. The factor loading of each of these items was above .5 and the total variable explained was 57.15%. Detailed information about the factor loading of each item is reported in Table  3 below. .70

Internal Consistency Reliability
To assess the reliability of the questionnaire items, we used Cronbach's α. The result of reliability of TLPACK, using the Cronbach's α of the full questionnaire, was .96. In terms of each construct, the Cronbach's α of technology knowledge was .91, learner knowledge was .88, pedagogy knowledge was .85, academic discipline content knowledge was .95, and context knowledge was .89. Based on these results, the TLPACK questionnaire developed was deemed to have good reliability.

Confirmatory Factor Analysis (CFA)
Before the CFA was undertaken, we adopted the suggestions of Bagozzi and Yi (1988) and Hair, Anderson, Tatham and Black (1998) and conducted an offending estimation to ensure the proposed model met the Preliminary Fit Criteria. Meanwhile, we also examined the goodness of fit between the proposed model and the data collected through model fit indices. Details of the preliminary fit criteria are presented in Tables 4 and 5 below:  The information conveyed in Tables 4 and 5 indicates that the error variance of each construct in TLPACK was positive and significant (p < .05). Furthermore, the standard error and correlation coefficient between constructs were less than 1 and the factor loadings were between .5 and .95. Based on these facts, the proposed model was in line with the preliminary fit criteria.
In terms of the model fit indices, the information is reported in Table 6. The model fit indices demonstrated that the proposed model has pretty good fit (χ 2 /df < .3, GFI > .8, AGFI > .8, IFI > .9, CFI > .9；PGFI as well as AGFI > .5 and RMSEA < .08). The indices revealed that, other than GFI, which was considered not good but acceptable (Browne & Cudeck, 1993;Sharma, 1996), the remaining indices confirmed that the model fit was quite good (Bagozzi & Yi, 1988;Bentler, 1990;Bentler & Bonett, 1980;Doll, Xia & Torkzadeh, 1994;Hayduk, 1987;Jöreskog & Sörbom, 1993;Segars & Grover, 1993). Subsequently, the Composite Reliability (CR) and Average Variance Extracted (AVE) were performed to examine the construct validity, which was discussed through the convergent validity and discriminant validity of indicators. The results of construct validity are reported in Table 7. The composite reliability indicated that the values of all five constructs were above the benchmark of .7 (Hair et al., 1998), which confirmed that the indicators of TLPACK had good composite reliability. As the convergent validity standardised factor loading of each TLPACK indicator was above .5, this met the standard suggested by Hair et al. (1998). The AVE of each indicator was between .50 and .62, and these values were also above the threshold of .50 set by Bagozzi and Yi (1988) and Fornell and Larcker (1981). Thus, the indicators of TLPACK developed in this study had good convergent validity. Hair et al. (1998) point out that discriminant validity of the indicators could be ensured by examining the correlation matrix among constructs. In other words, the square root of AVE of each indicator should be above the correlation coefficient of the other two constructs. The information provided in Table 6 shows that the square root of AVE of academic discipline content knowledge, technology knowledge, learner knowledge, pedagogy knowledge, and context knowledge was .78, .71, .72, .74, and .79 respectively, and that these values exceeded the correlation coefficient of the other two constructs in the matrix. Therefore, the indicators of TLPACK had good discriminant validity. Since the TLPACK had both good convergent and discriminant validity, it is sound to claim that it has good construct validity.

Results and Discussion
As pointed out by scholars, innovative changes in education should be initiated quickly (Cheng & Weng, 2017;World Bank, 2005). The application of modern technologies in the educational context requires of teachers to be equipped with a variety of knowledge in order to successfully orchestrate instructional activities (Evoh, 2009;Leendertz et al., 2013). Our research addressed an important issue regarding what types of knowledge modernday teachers need to acquire in the era of cloud pedagogy. We adopted the Delphi method to bring together experts' opinions and viewpoints on the expected knowledge. A TLPACK questionnaire was developed based on the results of the Delphi method. The TLPACK consisted of five constructs and 39 question items. The constructs interacting with each other are technology knowledge, learner knowledge, pedagogy knowledge, academic discipline content knowledge, and context knowledge. In this study, technology knowledge is defined as teachers' competence at integrating technology in their curriculum design and on-site instruction, which includes their ability to learn and operate new technology and the level of their sensitivity in relation to such use in/outside of class. Previous studies by Al-Senaidi, Lin and Poirot (2009), Goktas, Gedik and Baydas (2013); Leendertz et al. (2013), and Mofokeng and Mji (2010) suggest that even if teachers are able to use computers, their limited knowledge of technology hinders the effectiveness of integrating technologies into their pedagogy and hence it is advised that sufficient training is continuously offered to help teachers update their technology knowledge. This is highly expected to optimally integrate technologies in instructional activities (Du Plessis & Webb, 2008).
Learner knowledge refers to a teacher's ability to adjust his/her methods of instruction in accordance with various characteristics and genres of learners, in order to provide them with an adaptive learning experience. The importance of teachers' knowledge about learners and their learning effectiveness has been discussed in prior studies carried out by Hsu (2011) and Nielsen and Kreiner (2017). Pedagogy knowledge refers to teachers' competence and skills at planning, executing, and implementing instructional activities.
Teachers' knowledge in pedagogy influences the way they deliver the subject matter to learners, especially when there is a trend to integrate technologies into pedagogy (Baeten, Kyndt, Struyven & Dochy, 2010;Naicker, 2010).
Academic discipline content knowledge generally specifies the level of teachers' familiarity with the contents they are to deliver. Finally, context knowledge comprises teachers' ability to take up the context of their instruction and/or to finetune themselves to fit in the context. Academic discipline content knowledge has attracted attention from scholars such as Harris and Hofer (2017) as well as Swallow and Olofson (2017) who emphasise the importance of teachers' contextual knowledge in conducting successful educational activities.
Based on the results of data analysis, the TLPACK questionnaire developed in this study had good reliability and validity. It elicited 10 questions for technology knowledge, seven questions for learner knowledge, five questions for pedagogy knowledge, 12 questions for academic discipline content knowledge, and five questions for context knowledge. The proposed TLPACK scale is presented in Appendix A. This scale is being used in another larger scale study in Taiwan and the feedback we have collected on the applicability and usability of this scale has been quite positive.
Major limitations of this research are twofold. The first limitation is about the knowledge covered in this study which aims to provide teachers with pertinent information about what kind of knowledge they are expected to obtain to enhance teaching effectiveness. Due to the rapid development of modern technology creating innovative ways of learning and teaching, modern teachers should always extend their repertoire of knowledge. We suggest that future research can explore more knowledge that enables teachers to effectively and efficiently deliver the target content with a theoretical model of integrity, rigour, and objectivity. Additionally, the proposed TLPACK model aims to cover 31 pedagogical aspects engendered by five major facets and their interaction effects. Limited by factors such as manpower and time, this study only conducted in-depth discussion through the Delphi method and a survey research design, which led to the establishment of indicators for the five major facets. The relevant aspects of the interaction impacts are not covered in the study. It is recommended that follow-up research further elaborates on this limitation.

Conclusion
Teachers who strive to survive in the era of cloud pedagogy, which provides more affordable technology than before, are advised to acquire knowledge on technology, pedagogy, and content. This is why Mishra and Koehler proposed the concept of TPACK in 2006, and based on this conceptual model, other scholars considered more factors such as the development of technology and the learning environment to develop other models (ICT-TPCK and TPACK-XL). As all models have their positives and negatives, we integrated the dimensions covered by them and postulated TLPACK, which contains knowledge of various domains, e.g. technology, pedagogy, contents, learners, and context. The question items were developed by reviewing the relevant literature on the aforementioned models and then refined by five experts using six rounds of the Delphi method. The 39 questions were subjected to an exploratory factor analysis and a confirmatory factor analysis. Being data driven, the results of this study confirm that the indicators of the TLPACK constructed in this study have good reliability as well as validity. Through rigorous and robust exploration, the TLPACK theoretical facet indicators developed clearly reveal various types of knowledge and abilities pertaining to what modern teachers should master. When teaching, instructors are responsible for reviewing their own practices at any time with appropriate and effective tools in order to improve upon their weaknesses and strengthen their professional abilities. By doing so, teachers enhance their teaching efficiency and provide students with a better quality of teaching. TLPACK has been used to investigate teachers of a hospitality programme at high school level in Taiwan (e.g. Chen & Hsu, 2017) and the results of such an application were promising because these teachers had the opportunity to be introspective about their teaching quality and reinforce the knowledge they lacked. However, the above study had a fairly small scope, and a study with greater scope is currently being undertaken. Even so, the value of TLPACK has been acknowledged, and more empirical evidence is still needed.

Authors' Contributions
Liwei Hsu wrote the manuscript and Yen-jung Chen conducted all statistical analyses. All authors reviewed the final manuscript.