Computational Thinking Self-Efficacy Scale : Development , Validity and Reliability

The aim of this study is to develop a self-efficacy measuring tool that can predict the computational thinking skill that is seen as one of the 21st century’s skills. According to literature review, an item pool was established and expert opinion was consulted for the created item pool. The study group of this study consists of 319 students educated at the level of secondary school. As a result of the exploratory factor analysis, the scale consisted of 18 items under four factors. The factors are Reasoning, Abstraction, Decomposition and Generalization. As a result of applied reliability analysis, the Cronbach Alpha reliability coefficient can be seen to be calculated as .884 for the whole self-efficacy scale consisting of 18 items. Confirmative factor analysis results and fit indexes were checked, and fit indexes of the scale were seen to have good and acceptable fits. Based on these findings, the Computational Thinking Self-efficacy Scale is a valid and reliable tool that may be used in measuring to predict Computational Thinking.


Introduction
Computers have become an irreplaceable tool in most people's daily lives.Many realworld problems are solved by means of computers or using the principles of computer science.For this reason, an individual's ability to use computers effectively is a skill that may help them to manage problems in their day-to-day activities (Booth, 2013;Barr, Harrison & Conery, 2011).However, people in technologically-developed countries may experience difficulties in solving problems using computers if they do not have a robust technological background.These kinds of people may not know how to make their lives easier by using technology effectively and in a way that enhances their quality of life (Wing, 2008;Computer Science Teacher Association [CSTA], 2011;National Research Council [NRC], 2010).
At present, people living in a technologically-enriched society need three different skills in order to be able to keep up with that society.The first of these is the ability to use basic computer applications.This skill is generally defined as computer literacy.
The second is the ability to understand how computer systems work.This is generally defined as computer fluency.The third skill is the ability to use computer techniques or applications to solve specific problems.This skill is currently known as computational thinking (Perkovic and Settle, 2010).According to Wing (2006), computational thinking is a way of designing a system and understanding human behaviors by focusing on the basic concepts of the computer sciences.
Computational thinking as a problem-solving process promotes skills such as formulating problems, breaking down complex issues to find the main ideas and finding a solution using computers.While these skills are also used in problem-solving strategies other than computational problem-solving, a strong aspect of computational thinking is that it reduces the burdens on people through the use of technological devices (Booth, 2013).Nevertheless, computational thinking is not merely a skill used for solving problems involved with the programming of technological devices.Wing (2006) highlights the fact that computational thinking is not computer programming.In the same study, Wing also states that computational thinking does not mean "thinking like computers" and adds: "Computers are emotionless and boring; humans are smart and have strong imaginations.We humans make computers exciting." In cases when what Wing highlights is not considered, it is inevitable that computational thinking is perceived solely as programming.This perception supports the idea that "If we teach programming to people, their computational thinking skills will improve", and it conflicts with Wing's definition.Thus, one of the most important points to be considered is that computational thinking differs from programming skills (Qualls and Sherrell, 2010).While programming is a process used by computer scientists or programmers to solve certain problems, computational thinking does not only involve programming, but also an acquired point of view which understands the benefits provided by technology for solving problems.Thus, although computational thinking may be a key requirement in programming, it is not limited to computer science (Denning, 2009).
When computational thinking is treated as a problem-solving process, it can be seen that it contains other processes within it, such as breaking down problems and solving each part one by one and/or modeling the problem coherently (Astrachan, 2009).In a study conducted by Hong Qin (2009) highlighting computational thinking as problemsolving, it was concluded that students used common aspects of their thinking processes to solve different problems and that they understood the importance of thinking beyond problem-solving.The '21st century student standards' determined by the International Society for Technology in Education (ISTE) include high-level skills such as problemsolving, cooperation, creativity and critical thinking.According to Brichacek (2014), computational thinking is directly related to these standards.If we apply computational thinking to high-level skills such as problem-solving, creativity and critical thinking, our capacity for innovation is likely to increase.Given that innovation is one of the skills required for students in the 21st century, it can be stated that computational thinking has a crucial role to play in delivering the standards expected in this century.In this regard, computational thinking should be among the key 21st century skills (Philips, 2009;Wing, 2010).Hence, Wing (2006) states that, like literacy, computational thinking should be a skill that everyone possesses.A person with computational thinking skills will demonstrate the ability to: Understand which aspects of a problem may be solved with digital tools such as With the understanding of education changing all over the world, it has become the primary purpose of education to produce innovative, creative, problem-solving and productive individuals.It is believed that students should have skills that allow them to keep up with modern society.Computational thinking is one of these skills and is seen to be as much a part of education as reading and writing by some researchers, such as Wing; they also argue that it is a crucial skill (Bundy, 2007;Day, 2011).Given the recent increase in the number of studies on computational thinking and in the grants provided by different organizations for research relating to computational thinking, it can be concluded that this concept is a significant one.It is important that educators enable students to acquire computational thinking skills from the primary/elementary school stage rather than waiting until high school or university, so that students are able to keep up with the modern world (Qualls and Sherrell, 2010).
While attempts have been made to help students to acquire this skill in educational environments, measurement procedures and measurement tools are required to assess whether this skill has been acquired or not.In this regard, it is important to measure the concept of self-efficacy (Zimmerman, 1995) in relation to concepts such as academic success, motivation, educational self-regulation.Although it is known that a number of factors have an effect on the success of learners, self-efficacy and attitude have been shown to be more important than other factors (Austin, 1987;Anastasiadou and Karakos, 2011).Self-efficacy may be defined as how individuals perceive their present ability to use a skill or achieve a goal (Bandura, 1977).A belief in self-efficacy is an individual's belief about whether s/he can exhibit the behaviors necessary in order to reach a desired target (Akbaş and Çelikkaleli, 2006).When examined within the scope of social cognitive theory, self-efficacy is considered to have an important role in revealing the emotions which have an effect on an individual's performance (Bandura, 2001).Self-efficacy is directly affected by the choices people make and their cognitive and affective processes and motivation (Bandura, 1995, p. 5).People who believe that they will be successful are individuals with high self-efficacy (Bandura, 1995, p. 10).Individuals who believe in their abilities outperform other individuals in difficult tasks.A high level of self-efficacy increases the number of successful outcomes and the success-related happiness of individuals (Bandura, 2001).In other words, it may be said that self-efficacy positively affects success, despite not being sufficient to predict success by itself (Akkoyunlu and Kurbanoğlu, 2003).The main aim when measuring self-efficacy is to measure performances in carrying out a task, rather than measuring the personal features of individuals (Zimmerman, 2000).The measurement of self-efficacy may be regarded as a component which gives an idea of how successful individuals are likely to be.Measures may then be taken to enhance their success and performance (Askar and Davenport, 2009).Hence, the measurement of self-efficacy has come to be seen as important and a number of self-efficacy scales have been developed for many concepts in the educational field.Even, some scales related to computational thinking like programming were developed (Ramalingam and Wiedenbeck, 1998;Kukul, Gökçearslan, and Günbatar, 2017).These scales contain the items related to the programming languages or block programming.Since computational thinking is not programming (although it is related), computational thinking self-efficacy scale is needed.However, no scale was found for measuring computational thinking selfefficacy.It was thus necessary to develop a new scale for measuring computational thinking self-efficacy.

Method
The scale development steps of DeVellis ( 2003) were followed while constructing the Computational Thinking Self-efficacy Scale.The first step is deciding what is to be measured.As computational thinking itself is a high-level skill, it is hard to observe.However, observing the skills that may predict this skill, and measuring the perception of a person relating to the presence of these skills, may provide easier and more correct results.Hence, it was decided to measure computational thinking self-efficacy.It was found that no such scale had been developed for measuring computational thinking selfefficacy.Therefore, a new scale was required.

Creation of the Item Pool
Item pool creation was the second step in developing the scale.The literature was reviewed in order to create an item pool, and definitions of computational thinking made in the literature and research on computational thinking were reviewed.The studies of Wing (2006Wing ( , 2008)), who enabled the popularization of the concept in the second half of the early 2000s were also examined.In addition to these studies, the definition of computational thinking given by the CSTA and ISTE and its sub-dimensions as well as the definition given by the organization Computing at School (CAT) were taken into consideration.At the end of the necessary research, an item pool of 47 items was created.The items were sent to 11 experts to get their opinions.6 of the experts were academics working in programming/coding education in the field of Computer and Teaching Technologies Education.Among the other experts, one was a language expert, one was an assessment and evaluation expert and three were teachers working in the Ministry of National Education.All items of the scale and the factors predicted for the items, as well as a form outlining the purpose, scope and target group of the scale were sent to the experts via e-mail in order to gain their opinions.The experts were asked to assess each item as "appropriate", "inappropriate" or "improvable", and to add an explanation in a separate section for the "inappropriate" or "improvable" items.In addition, a section was incorporated below the form in which the expert could add items, and the experts were asked to add any suggestions for items in this area.
The items were re-examined in accordance with the opinions from the experts.The items deemed to be proper to exclude from the scale were removed, and the items suggested were incorporated.Consequently, a 51-item Self-efficacy Scale was obtained which was arranged as 5-point Likert type (1 = "Completely Disagree", 2 = "Disagree", 3 = "Neutral", 4 = "Agree", 5 = "Agree Fully").The scale was applied as a pilot to secondary school students who would not participate into the real application in order to check age level conformity.Then minor revisions were made in the items for points not understood.

Participants
During the scale development stage, 409 students who were at secondary school in the 2016-2017 academic year were targeted.The students with data losses and who did not have any programming/coding education were excluded from the analyses.The analyses were conducted on 319 students.Demographic information relating to the study group is given in Table 1 and Table 2. On examination of Table 1 and Table 2, it can be seen that 153 (48%) of the students participating in the study were female and 166 (52%) were male.Considering the grades of the students, it can be seen that 118 (52%) were in the 5th Grade, 189 (59.2%) in the 6th Grade, and 12 (3.8%)were in the 7th Grade.Examining the students' programming/ coding education it can be seen that 223 students had had programming/coding education with Scratch and 115 students with Code.orgeducation.The number of the students educated in Small Basic was 15, the number of the students educated in Alice was 7, and the number of the students educated with tools other than these tools was 7. The reason that the sum of these numbers exceeded the total number of the students was that students had been educated in more than one programming/coding tool.

Results
Exploratory factor analysis was performed in order to determine whether the scale was reliable.The results of the Kaiser-Meyer-Olkin (KMO) and Bartlett Sphericity tests were examined in order to determine whether the data obtained from the scale was appropriate for exploratory factor analysis.The KMO value was found to be .936,and this value may be interpreted as showing that the conformity of the data structure was suitable for performing the factor analysis in terms of sample size (Leech, Barrett & Morgan, 2005;Şencan, 2005;Tavşancıl, 2005).On examination of the Bartlett test results (X 2 : 6992.560,SD: 1275, p < .01), the obtained chi-square value was seen to be significant at the .01level.This result indicates that the data came from multivariable normal distribution and thus the factor analysis could be continued.
Principal components analysis was selected as the factorization method and maximum variability (varimax) was selected from the vertical rotation methods with the aim of revealing the factor pattern of the Computational Thinking Self-efficacy Scale.

Results Relating to Validity
On examination of the item statistics as a consequence of the conducted factor analysis, the items that seemed to have close factor load values in more than one factor (Kline,1993;Tabachnick and Fidell, 2013) or items with item load values detected to be under .30were exclude.The factor analysis was then repeated, and an 18-item form was obtained.
At the end of the analysis, it was seen that there were four components with an eigenvalue over 1 for 18 items essential for the analysis.The contribution of these components to the total variance was found to be 54.717%.The said four components were seen to contribute significantly when the importance of their contribution to total variance was considered by examining both the explained total variance table and the scree plot graph.Accordingly, a repetition of the analysis was not required.
The results of the Kaiser-Meyer-Olkin (KMO) and Bartlett Sphericity tests were examined again in order to determine whether the data obtained with the 18-item form was appropriate for exploratory factor analysis.The KMO value was found to be .904,and this value may be interpreted as showing that the conformity of the data structure was perfect for performing the factor analysis in terms of sample size (Leech, Barrett & Morgan, 2005;Şencan, 2005;Tavşancıl, 2005).On examination of the Bartlett test results (X 2 : 1778.892,SD: 153, p < .01), the chi-square value obtained was seen to be significant at the .01level.This result indicates that the data came from multivariable normal distribution and that the factor analysis may thus be continued.At the end of the factor analysis conducted with 18 items, results relating to the variance explained by the components were given in Table 3.
On examination of Table 3, it can be seen that the contribution of the factors to the total variance after rotation was 15.19 for the first factor, 14.72 for the second factor, 12.47 for the third factor and 12.33 for the fourth factor.
In the exploratory factor analysis, which was conducted with the aim of revealing the factor pattern of the Computational Thinking Self-efficacy scale, the acceptance level for the factor load value was determined to be .40.Accordingly, it was observed that no item had a factor load value under .40.The factor pattern relating to the scale, factor load values of the items and common factor variances are given in Table 4.As seen in Table 4, the factor load values in sub-scales level varied between .569 and .781for factor 1, between .478 and .731for factor 2, between .546 and .764for factor 3 and between .601 and .750for factor 4. On examination of the factor load values in terms of magnitude, it is possible to classify the load values of the items except m50 from "good" to "perfect".Item m50 may be classified as "medium" (Comrey and Lee, 1992).Kim-Yin (2004) highlighted that sample size should be considered when deciding to exclude an item from the scale.Accordingly, an item with a 0.40 factor load should have a sample size of at least 200 (cited by Şencan, 2005).That there were 319 participants in this study shows that an adequate sample size was reached for m50.Haynes, Richard and Kubany (1995) stated that one of the most frequently used methods for excluding inappropriate items from a scale, enhancing content validity and determining the validity of its scope was expert opinion.At the end of the analyses con- .601.439 Table 5 Examples from Items in the Scale

Factor Item
Reasoning I can decide whether the data to be used for the solution of the problem is adequate or not Abstraction I can make comments on the data used for the solution of the problem Decomposition If there are sub-problems in the problem, I can manage the solution processes of these subproblems Generalization I can make connections between the current problem and previously encountered problems ducted, expert opinions were sought for naming the factors and to enable scope validity.The naming of the factors presented was accepted by the experts.Accordingly, the factors were labeled "Reasoning", "Abstraction", "Generalization" and "Decomposition".In terms of scope, the 18-item form was considered to enable scope validity for the age level.Table 5 presents item examples from the scale.

Results Relating to Reliability
Prior to the reliability studies of the scale, the results of the validity studies were examined.After sorting out the overlapping items and items with item load under .30,reliability studies were conducted with the remaining 18 items.In order to determine the distinctiveness of the items in the four-factor scale, the t-test was conducted for determining whether the difference between test and item mean points of the lower and the upper 27% groups was in accordance with the total points.On examination of test and item means of the lower and upper 27% groups, it was seen that the difference between the lower and the upper 27% groups in the whole test was significant.Also, on calculation of item mean points, it was seen that the mean points of the lower and the upper 27% groups differed significantly.Cronbach's alpha reliability coefficient was used to test the internal consistency of the scale, and the reliability of the whole scale and the sub-factors were calculated.Reliability coefficients calculated for each factor and the whole scale are given in Table 6.
On examination of Table 6, as a result of applied reliability analysis, Cronbach Alpha reliability coefficient can be seen to be calculated as 0.884 for the whole self-efficacy scale consisting of 18 items; 0.772 for the Reasoning sub-scale consisting of 5 items; 0.774 for the Abstraction sub-scale consisting of 5 items; 0.701 for the Decomposition sub-scale consisting of 4 items; and 0.718 for the Generalization sub-scale consisting of 4 items.

Results Relating to Confirmatory Factor Analysis
The fit of model relating to the factor structure revealed as a result of Exploratory Factor Analysis (EFA) was tested with Confirmatory Factor Analysis (CFA).Conformity of the obtained model was tested with χ2/sd, RMSEA (Root Mean Square Error Ap- proximation), NFI (Normed Fit Index), NNFI (Non-Normed Fit Index), RMR (Root Mean Square Residual), CFI (Comparative Fit Index), IFI (Incremental Fit Index), GFI (Goodness of Fit Index) and AGFI (Adjusted Goodness of Fit Index) fit criteria.Required modifications were made in accordance with the analysis results, it was detected for the fit of model that χ2/sd = 1.535;RMSEA = 0.041; NFI = 0.892; RMR = 0.049; CFI = 0.959; IFI = 0.959; GFI = 0.959 and AGFI = 0.916.For a goodness of fit index, 0 indicates that there is no fit between the data and the model, and 1 indicates that there is a full fit.When the value of an index is greater than 0.90 and close to 1, it may be said that fit in the data is almost enabled (Çerezci, 2010).Şimşek (2007) states that an χ2/sd value of or under 5 and an RMSEA value of or under 0.08 indicate good fit.Byrne (1998) states that RMR and SRMR values of or under 0.10 are required for good fit.Again, an IFI, CFI, NFI and NNFI greater than 0.90 indicates a good model.In addition, an AGFI of or greater than 0.80 and a GFI of or greater than 0.85 indicate acceptable fit (Çokluk et al., 2010).Meydan and Şeşen (2011) utilized fit values indicated by different research in their study and tabulated statistical values relating to model fits (p.37).Accepted intervals in accordance with this table and model fit values in accordance with the data obtained are given in Table 7.
On examination of Table 7, it can be seen that the Chi-Square Fit test result showed good fit.Fit indexes indicate that indexes other than NFI and CFI exhibited good fit.CFI is seen to be within the acceptable fit interval.As the NFI value was so close to the acceptable fit border, no modification was required.
There are different opinions relating to which fit indices are to be reported in the confirmatory factor analysis.While there is an agreement on reporting the chi-square fit test (Meydan and Şeşen, 2011;İlhan and Çetin, 2014), it is also deemed beneficial to state the results of a few other indexes (Meydan and Şeşen, 2011, p. 38).In this respect, the fit values reported in this study may be considered adequate.On examination of the goodness of fit indexes obtained in this study, the model fit of the scale is seen to be at a statistically acceptable level.The model formed as a result of amendments relating to the scale developed and the relations between the factors are given in Fig. 1.On examination of Fig. 1, it can be seen that the standardized relationship values between factors were .60 and over.On examination of the relationship between items and factors, it was determined that values ranged from .50 to .75.

Conclusion
At the end of this study, a scale was developed which consists of 4 factors and 18 items for measuring the computational thinking self-efficacy of secondary school students.While developing the scale, the items were formed by utilizing the related literature and expert opinions.The scale was finalized with the application of exploratory and confirmative factor analyses.According to the results of the exploratory factor analysis, the scale explained 54.717% of the total variance.According to the reliability study conducted, the reliability coefficient of the whole scale was detected to be .884.
The reason behind the decrease in the number of items in the final form of the scale compared with the initial ones might be the close relationship between the CT concepts.Wing (2008) emphasizes that one of the two skills that is important for CT is Abstraction.For example, creating algorithms is a kind of abstraction.This result explains the absence of an algorithmic thinking factor on the scale.Because of the factor loadings of the items categorized under algorithmic thinking were higher in the abstraction factor, no separate algorithmic thinking factor was formed.A similar result was also observed for evaluation factor.According to the Computing at School (2014), although Evaluation is a sub-dimension of Computational Thinking, a separate factor was not constructed as the evaluation was carried out at every step of Computational Thinking.When the items classified in the first factor are analyzed, it was observed that these items are related with algorithmic thinking, evaluation, and the data that can be used for problem-solving.Cognitive factors are also known as effective in explaining CT skills.One of these factors is reasoning ability (Roman-Gonzalez, Perez-Gonzalez, Moreno-Leon, & Robles, 2018).When the items collected in the first factor were evaluated, it was seen that they were suitable for reasoning ability.At the end of the literature review, the sub-dimensions of the scale were given the headings "Reasoning", "Abstraction", "Decomposition" and "Generalization".It was decided that this nomenclature was appropriate in accordance with opinions from the experts.
Confirmatory factor analysis results and fit indexes were checked, and fit indexes of the scale were seen to have good fits, other than NFI (χ2/sd = 1.535;RMSEA = 0.041; NFI = 0.892; RMR = 0.049; CFI = 0.959; IFI = 0.959; GFI = 0.959; AGFI = 0.916).The fact that the NFI value was close to .90, the acceptable fit value, was interpreted as meaning that it could be ignored and modification was not required.These results can thus be interpreted to show that the scale was theoretically appropriate.
The scale that has been developed is considered to be of benefit for the measurement of computational thinking, a topic which has been intensively studied by educators recently.Considering the relationship between CT self-efficacy and CT skills (Roman-Gonzalez et al., 2018), this scale can be considered as a measurement tool that can help teachers develop their students' CT skills.Its structure validity could be tested by applying the scale to more secondary school students.In this way, a more valid scale could be obtained.In future research, a similar measurement tool with different items and for different age levels could be assessed.Studies could also be conducted on the relationship between different measurement tools developed to measure computational thinking and this self-efficacy scale.Results of this research may enable modification of the measurement tools developed and more accurate measurement of computational thinking.

Table 1 Demographic
Information of Self-Efficacy Scale Study Group

Table 4
Computational Thinking Self-Efficacy Scale Factor Pattern

Table 6
Reliability Coefficients Relating to Computational Thinking Self-Efficacy Scale

Table 7
Statistical Values Relating to Structural Equality Model Fit and Fit Points of the Scale