Turkish Adaptation of Online Teaching Effectiveness Scale

With this study, it is aimed to adapt the Online Teaching Effectiveness Scale into Turkish Language. The adaptation process has been done within four study groups. The first group took part in the study after having the Turkish form of the scale as required according to the translation steps. The study has been done with the MCBU Faculty of Education during the 2021 Spring Term. With the first study group, language validity was ensured with 30 participants by applying the Turkish and English versions of the scale in an interval of two weeks. According to language validity findings the Spearman’s Rank Order Correlation value has been found above .70. Then pilot study has been done with second group with 62 participants. The total correlation values of the items are between .55 and .80. The third group consisted of 436 participants and after second run the CFA results are found like that; χ² / df = 3.582, RMSEA = .80, SRMR = .35; CFI = .95; TLI = .94, NNFI = .95. With the third study group the validation study has been finished and continued to reliability study. The fourth and the last group which consisted of 96 participants and the applications has been done in an interval of three weeks. According to the Cronbach’s Alpha values each of the dimensions and the total of the scale have points between .70 and .97. The test – retest values are between .72 and .94. Based on the findings the Turkish version of the scale is valid and reliable for usage.


Introduction
The concept of distance learning can be defined as a flexible, contemporary, and effective learning style that largely removes time and space limitations and allows the use of different technologies in educational environments. Looking at the historical background of the concept, three main phases can be given. According to Jones (1996), the first of these is the traditional phase where student -teacher communication is achieved through correspondence. The second phase is known as industrialized multimedia education and has a sound and video predominant structure. The last phase is the period when computers and modem are used in education.
At the point where technology has come to an end, it is seen that it has two important values such as computer and internet in teaching environments. When these two important values are brought together, they make many approaches that can be an alternative to physical space -oriented education such as school and classroom meaningful (Tuncer & Taşpınar, 2008). The basic starting point of online education, which comes to life because of bringing the internet and computer together, is the phenomenon of interaction. In short, the fact that the source is the receiver, and the receiver is the source is the essence of the concept of interaction. The interactive teaching of the lessons between the teacher and the student has created interactive teaching technologies and therefore the internetbased distance education model has become an important service that is widely used today (Rovai & Barnum, 2003).
While these changes in online education create changes in many aspects of the teaching process, especially the design and implementation stages of educational programs are also greatly affected by these changes. Considering the dimensions of the curricula, it can be said that these changes affect the teaching process dimension of the curricula, but also on other dimensions (objectives, content, instructional methods, and assessment) that are connected to each other with a dynamic structure.
courses are not suitable for online education, feedback and corrections are not done efficiently during lessons, interaction problems and technological infrastructure problems in groups with many students.
In addition to these problems in online education, one of the issues that should be addressed in the process is the evaluation of online education practices. At the end of the teaching -learning process, whether the targeted behaviors are achieved or not, in other words, revealing the effectiveness of the programs implemented, corresponds with the evaluation processes. As a result of these evaluations, the negative and positive aspects of the implemented program are revealed, and necessary corrections and improvement studies are made on the defective directives of the program.
The main purpose of the evaluation activities is to reveal whether the planned and applied desired level is successful or not. Since online education consists of a process where students and teachers do not share the same environment, evaluation activities should be carried out considering this disadvantage.
Since the presentation of testing tools and receiving the expected feedback is different from that of face -to -face training, it is necessary to anticipate some negative consequences that may arise in the evaluation of online training and to flow the necessary precautions. Especially the fact that the testing tools used in the evaluation of the process consist of modern tools (projects, portfolios, performance assignments, etc.) rather than traditional tools will provide more realistic data for determining the effectiveness of the curricula.
Whether you train online or face to face, one of the variables that greatly affects the effectiveness and quality of teaching activities is the competence of teachers in this area. Especially the knowledge, skills and experience of the teacher who teaches the course in online education regarding distance education applications are of great importance not only for a successful classroom management but also for the effectiveness of the education provided (Seufert, Guggemos ve Sailer, 2021). The self -efficacy perceptions of teachers participating in online education regarding the process and themselves can also be shown among the factors that greatly affect the effectiveness of the process. Studies in this field by Ottenbreit -Leftwich, Glazewski, Newby & Ertmer (2010); Türel & Johnson (2012); Cabı (2018); Vangrieken, Packer & Kynd (2017) reveals that teachers do not consider themselves competent in online education, so this situation poses a problem in terms of the quality of the education provided. Undoubtedly, teachers' belief in their own competences is closely related to their ability to use online teaching environments and to achieve the goals of the course.
Teachers' involvement in the online education process in an active and motivating way for the students, guiding them in the process. expected. Their responsiveness to these expectations is related to the communication skills of teachers, their different teaching methods and technical knowledge, and their competencies in their fields. Teachers' efforts to increase their knowledge, skills and attitudes in these areas will have a positive effect on their professional development, their selfefficacy perceptions, and thus on the effectiveness of online education practices.
The competencies of teachers in evaluating the quality of online education can be discussed under some headings. These titles can be listed as teacher presence, teacher expertise, facilitation, and engagement. On the other hand, considering the areas of competence that teachers should have in the online education process, the presence of the teacher in the field and in the classroom is important.
The term presence here; teachers' sharing of their knowledge and experiences in their field of expertise with their students and using their creativity to give original and striking examples related to the subject can be defined as the skills to arouse curiosity and interest in the course in students thanks to their high motivation. With the mentioned teacher presence, it can be ensured that some limitations in the natural structure of online education are minimized and the course objectives are achieved at the desired level.
One of the competences that affect the quality of online education and should be found in teachers is expertise. Specialization here includes the teacher's expertise in his subject area, as well as classroom management and information technology use skills. It is useful if they can plan the teaching process, express what they want from the students, and establish a relationship based on respect and love with the students in the process, as well as their command of the subject areas. Competencies such as the ability to use information technologies, which are one of the requirements of online education, sharing course materials on the web, associating visual and auditory course materials with acquisitions, and providing multiple learning environments, can be shown among the behaviors that teachers should have at the level of expertise in online education.
The success of the teaching process, in other words, one of the most important conditions for achieving the course objectives is the planning of the process. The ability of online education teachers to plan and apply the process is important in eliminating some of the problems encountered in distance education. Continuing the process effectively and efficiently requires a good planning skill to save both material and moral. Teachers' displaying a facilitating attitude to the process, clearly revealing what they want from their students and guiding them in the process can be shown among the skills that teachers should demonstrate in online education.
In online education, student -teacher interaction is one of the issues that should be emphasized. The fact that students and teachers do not physically share the same environment in the learning process increases the importance of the concept of interaction. In cases where there is not enough interaction, students may face the situation of losing their motivation for the lesson. For this reason, the interaction of online education teachers with their students in the process is of vital importance in terms of achieving the goals of the course. It is one of the qualifications that teachers who provide online education should have to answer their students' questions simultaneously or synchronously and offer them the process of reflection and correction.
Reyes -Fournier, Cumella, Blackman, March & Pedersen (2020) based the theoretical foundations of the scale of the effectiveness of online education on the seven basic principles of undergraduate education stated in Chickering & Gamson, (1987) and the work of Thomas & Graham (2017). It is seen that the scale of effectiveness of online education consists of presence, expertise, facilitation, and interaction sub -dimensions. While the items related to the existence subdimension of the scale indicate that teachers who teach online have high interest, good presentation, meaningful examples, and high motivation, it is seen that the items related to the field of expertise include their field dominance and respect for the student. In the facilitation sub -dimension of the scale, it is seen that it consists of items related to teacher planning and explicitly presenting what is desired, while the interaction sub -dimension consists of items for providing offline or online interaction and providing instant feedback processes. Just as not planning education and training activities is very important, it is also important to implement the planned process effectively, in other words, to reach the goals of the course. For this reason, studies aimed at eliminating the limitations caused by determining the factors that have a negative impact on the effectiveness of online education are considered valuable in terms of both the effectiveness of the process, the degree to which the education reaches its goals, and all the economic and labor expenditures made for the process.
The aim of the study is to adapt the "Online Teaching Effectiveness Scale" developed by Reyes -Fournier, Cumella, Blackman, March & Pedersen (2020) into Turkish Language. In this meaning adapting the original scale into Turkish language has been occurred as the problem situation while these statements below have been determined as sub -problem statements of the study: According to the "Online Teaching Effective Scale" 2. What are the language validity study findings of its Turkish version?
3. What are the pilot study findings of its Turkish version? 4. What are the validation study findings of its Turkish version? 5. What are the reliability study findings of its Turkish version?
By the guidance of these sub -problem statements, the adaptation study has been done as explained in the methodology part.

Method
With this part the information about the original scale has been given, the process of adapting the scale has been explained, the study groups have been expressed, and the analyzing processes applied during the adaptation process has been explained.

Online Teaching Effectiveness
Online

Process of Adapting the Scale
The process of adapting the scale has begun with the current developments. As it is known, because of the pandemic in the world, the need for distance education has increased and the need for a measurement tool to evaluate the effectiveness of the distance education process has been felt by researchers. Researchers have focused on a scale which is actual, effective, short, user friendly and developed by the experts from this area. Focusing on this direction has carried the researchers to a scale called OTES (Reyes -Fournier, Cumella, Blackman, March & Pedersen, 2020) as introduced in the related topic above. First, permission to use the scale was obtained from the developers via email.
Then the permission process had been completed by getting permission from the MCBU Faculty of Education and MCBU Social Sciences Ethics Commission and adapting scale process has been continued with technical issues. During the adaptation of the scale into Turkish language five main steps has been fallowed as Erkuş (2012), Karakoç & Dönmez (2014) and Seçer (2015) had offered.
The steps can be called as translation study, language validity study, pilot study, validation study and reliability study. Scale adaptation process has been started with translation study. Translation study has been applied as Beaton, Bombardier, Guillemin, & Ferraz (2000) guided. For a cross -cultural adaptation process Beaton, Bombardier, Guillemin, & Ferraz (2000) has offered the steps as translation, synthesis, back translation, expert committee review and presenting as the translation steps. The collected data after translation study has been analyzed with SPSS 25 and Amos 24 packet programs. The adaptation process of the scale has been applied in this meaning and four study groups had been formed for this process.

Study Groups
Turkish language adaptation process of the scale was carried with four main study groups.
Groups were formed with the attendance of MCBU Faculty of Education undergraduates in 2021 spring term. According to Seçer (2015) 30 participants are enough for the language validity studies.
With the first group which had 30 participants language validity was ensured. Second group, which had 62 participants, contributed the pilot application. Confirmatory factor analysis was conducted to test the construct validity with the third group of 436 participants. The test -retest study was conducted with an interval of three weeks with the fourth group consisting of 96 participants formed to test the reliability. The 30 participants of language validity application were from Guidance and

Analysis
The analysis of the data set during the adaptation process has been applied with in four main steps after language translation study. Before beginning the analysis of the data, the translation process had been done, which was named as translation study, and the Turkish translated form had been reached in this way. Then the collecting of the data set had been carried out by the help of Turkish version of the original scale and the analysis of the data had been done with the participant answers of that form.
In the first step language validity had been tested to see the linguistic equivalence. As mentioned under the sub -title, where the study groups had been explained, first study group had been formed for testing the language validity. With this study group Turkish and English forms of the scale had been applied with an interval of two weeks. Firstly, the collected data from these applications was tested if they had been distributed normally or not. It was seen that the data wasn't distributed normally, and Spearman's Rank Order Correlation Test was used, as a nonparametric test, to see if there was a correlation between Turkish form and English form of the scale. Tabachnick & Fidell (1996); Byrne (2010) have told that for mentioning existence of a correlation between groups, there must be above .70 correlation between groups. With the results of the test, it was seen that there was high correlation between the groups as shown at the findings part of the study.
The second step of the analysis was applied with the second study group which had 62 participants. In the second step, the pilot scheme was conducted with the Turkish form of the scale which is tried adapting here. The pilot scheme had helped to observe the values of total correlation of the scale items and the Cronbach alpha internal consistency value. It was told that the values of total correlation of the scale items should have been above .30 and the Cronbach alpha value should have been above .70 according by Büyüköztürk (2012) and Seçer (2015). This situation has been observed as shown at the related part of findings.
The third step is about the confirmatory factor analysis. With the data obtained from the third study group, which had 436 participants, the confirmatory factor analysis has been done after the outliers and extreme values of the data set had been removed. After removing the extreme values from the data set, the analysis was made with the data of the remaining 406 participants. The data of thirty participants had been removed for that issue. Confirmatory factor analysis had helped to observe confirming the construct validity of 12 items and 4 factors as in the original scale and the confirmatory factor analysis results has been shown at findings part.
In the fourth and the last step of the analysis reliability studies have been done. With an interval of three weeks the test -retest study had been conducted. The data obtained from the fourth group, which had 96 participants, was analyzed, and was tried to see the Cronbach alpha internal consistency coefficients of the sub -dimensions and the whole of the scale.
So, in four steps the analysis has been applied and the results have been explained in the following part by the help of these four analysis steps.

Results
In this part the findings of translation study, the findings of language validity study, the findings of pilot study, the findings of confirmatory factor analysis study and the findings of reliability study has been shared.

Translation Study
For the first step of translation study, the process has been started by the help of three experts from the field of educational sciences who are good at English language. According to Savaşır (1994) the translation studies should be done with the experts who are good at both two languages and are aware the subject of the study. Later, experts came together, compared their translations, and combined their individual translations into a single form. Then, the translated version of the scale as a single form was sent to five academicians, who were experts in the field, and opinions were taken on the comprehensibility and scope of the translation. In line with the opinions of five academicians, arrangements were made on the translation. For the second step, thirty students who were good in English were put into practice with this version of the translation. The Turkish and English forms of the scale were applied with an interval of two weeks and the correlation between them was examined.
With this application, it was checked whether there was a semantic problem in this state of the scale.
For the third step, back translation was done with two professional translators individually. Then a meeting was organized, they compared their translations and put their translations into one form. With two native speakers of English language the back translated form and original form were compared, and the translation phase was ended. For the fourth and fifth step, the opinions of three academics, who have mastered the English language and the related methodology, were taken about all the stages and reports up to this moment and the final version of the form was decided and presented.

Language Validity Study
Language validity study has been done with 30 participants who were good at English as a foreign language. Seçer (2015) has said that 30 participants are enough for language validity studies.
So, this discourse has been supplied with the number of participants in this part of the study. The application of English and Turkish forms has been done with an interval of two weeks and because of not showing normal distribution of the data, the correlation test has done according to Spearman's Rank Order Correlation Test which is known as nonparametric one of Pearson Test. The findings of the data according to Spearman's Rank Order Correlation are r = .724, p< .05 which means above .70 as a necessity. According to the findings of the language validity analyzes it can be said that both Turkish and English forms of the scale are equivalent.

Pilot Study
After supplying language validity pilot study has applicated as the next step. For a pilot study the participant number should be two or three times of the scale items. We have twelve items, and the participant number is sixty -two at this step of the study. According to the findings of the pilot study, the data is shown in Table 1.

Validation Study
During the cross -cultural adaptation of a scale doing Confirmatory Factor Analysis instead of Exploratory Factor Analysis is a common approach. With the CFA a current scheme can be tested.
By the help of CFA researchers try to confirm the existence structure of a scale in its adapted version.
CFA tries to test a prediction that variables will take place on predetermined factors based on a certain theory. While Büyüköztürk (2012) and Seçer (2015) offering CFA for adaptation studies, they also say at least above 300 participants would only be enough for the validation of the adapted scale. In this step of the study four hundred and thirty -six participants had joined to data collecting process but thirty of the total data were deleted because of being outliers and having extreme values. So, the analyzes had been done with the data of four hundred and six participants. As Meydan and Şeşen (2015) offered CFA should be done at least in two levels during a process of a scale adaptation study, two times CFA has been done and the findings of each of the steps have been shown in Figure 1 and  In Figure 1, the first level CFA shows the structure of the Turkish Language adaptation version as same as the original study with in 12 items and 4 four factors. According to Kline (2005) and Brown (2006) chi -square, RMSEA, CFI, SRMR measures are the most important points while reporting the findings of CFA. The first level CFA model fit values are as shown in Table 2. With the first level CFA study the measurements were mostly obtained as accepted (Kenny, 2020). The values have been found as offered; χ²/df is under 5.00, SRMR is between 0 and .08, CFI and NNFI are above .90, but RMSEA has been above .80 in the first level analyses. Then the good fit indices have been reached as shown at Figure 2. In Figure 2, the second level CFA shows the structure of the Turkish Language adaptation version as same as the original study with in 12 items and 4 four factors. At the second level by drawing just two covariances between item 3 and 4 / item 4 and 6, the good fit values were preserved and RMSEA value has been reduced at .80 as Hu & Bentler (1999) had pointed out. The second level CFA model fit values are as shown in Table 3. Looking at the findings in Table 3, it can be said that good fit indices were achieved with the two -stage intervention performed within the scope of the CFA stage of the study. The findings in Table 3 show that the scale adaptation has sufficient fit indexes at this level and the validation has been supplied.

Reliability Study
For the reliability of OTES Turkish language adaptation form, the test -retest has been done, Cronbach alpha internal consistency coefficients has been reached for the scale sub -dimensions and the whole scale. The reliability study was conducted with the fourth study group which had 96 participants which was eight times the number of the items. The study was conducted three weeks apart and the findings shown in Table 4 were achieved. As seen in Table 4, the Cronbach's alpha values of the general and sub -dimensions of the OTES Turkish language adaptation version show that the items of the scale have sufficient internal consistency. The Turkish language form had been applied to the same study group with an interval of three weeks and it has been seen that the correlation values between the scores obtained from both applications are above .70 for the whole scale and sub -dimensions of the scale. Having above .70 values in Cronbach's alpha and Test -Retest findings shows the reliability of the OTES Turkish language adapted scale.

Discussion, Conclusion and Recommendations
The form of education known as distance education or online education is gaining increasing importance in today's world with today's experiences. This study aimed to adapt the "Online Teaching Effectiveness Scale" (Reyes -Fournier, Cumella, Blackman, March & Pedersen, 2020) into Turkish.
In the adaptation process, five main steps, which are stated as translation study, language validity study, pilot study, validation study and reliability study, were followed. During the translation study Beaton, Bombardier, Guillemin, & Ferraz's (2000) views has been taken as the guide. Following the completion of the translation study, the other stages of the study were started and as can be seen in the findings, a valid and reliable measurement tool has been introduced into Turkish language for evaluating the distance education process.
As the last step, the reliability study was carried out with 96 participants, who were eight times the number of the items in the scale, within interval three weeks. According to reliability study the values of Cronbach's Alpha and Test -Retest are above .70 for all sub -dimensions and for whole of the scale.
So, as a conclusion the adaptation study resulted in obtaining a valid and reliable scale. The Turkish version of OTES has four sub -dimensions as same as been in the original one. The twelve items of the scale can evaluate online education in a fast, simple, user friendly and effective way as the developers of the original scale has claimed. The adaptation version of OTES has been named as "Online Öğretim Etkililiği Ölçeği" into Turkish.