An Exploratory Study to Assess Analytical and Logical Thinking Skills of the Software Practitioners using a Gamification Perspective

The link between analytical and logical thinking skills and success of software practitioners attracted an increasing attention in the last decade. Several studies report that the ability to think logically is a requirement for improving software development skills, which exhibits a strong reasoning. Additionally, analytical thinking is a vital part of software development for example while dividing a task into elemental parts with respect to basic rules and principles.  Using the basic essence of gamification, this study proposes a mobile testing platform for assessing analytical and logical thinking skills of software practitioners as well as computer engineering students. The assessment questions were taken from the literature and transformed into a gamified tool based on the software requirements. A focus group study was conducted to capture the requirements. Using the Delphi method, these requirements were discussed by a group of experts to reach a multidisciplinary understanding where a level of moderate agreement has been achieved. In light of these, an assessment tool was developed, which was tested on both software practitioners from the industry and senior computer engineering students. Our results suggest that individuals who exhibit skills in analytical and logical thinking are also more inclined to be successful in software development.


Introduction
We now live in a world where new technologies are frequently introduced.To be successful in this competitive environment, an individual must be able to improve his or her cognitive skills every day.In such a world, knowledge workers need to utilize the collected knowledge and use it to develop better skills in an effective way.However, improving personal and professional success depends on lifelong learning and continuous self-improvement [1].
In particular, students of this new area should understand the relationship between their analytical and logical thinking skills and their future success.Jacobsen et al. [2] suggested that there is a positive relationship between sensory and cognitive attributes of individuals, and ultimately the success level of students is directly proportional to such skill sets.In a study conducted in 17 countries, Bloom [3] also argued that the cognitive attributes of an individual are directly proportional to his academic success.The thinking abilities of individuals were accepted as a skill and different definitions were made.Lipman [4] stated that thinking skills vary from person to person since they are personal skills.He also argued that differences could arise in comprehension and discernment as logical thinking skills differ based on individuals.
Improvement of individuals' logical and analytical thinking skills has become important because it affects the success of individuals in the field of software engineering.The basis of logical thinking is the sequential thinking process.The continuation of the process requires the ideas, findings and the results related to problems to be determined and the data obtained to be regulated.The success of reaching the goals and dealing with the difficulties of the complex world depends on the logical thinking skills of the individuals [5].One of the subjects put excessive emphasis on in education is logical thinking ability and it is indicated that it has a great effect on student's success [6].
Sternberg and Grigorenko [7] divided thinking skills into 3 categories: analytical, creative and practical thinking.Analytical thinking involves the processes of knowledge-based problem solving and decisionmaking.According to a definition provided by Ruskin [8], being analytical is "breaking things, situations, practices, problems, statements, ideas, theories, arguments down into their component parts."[8, pp. 1], Mathematics is the foundation of analytical thinking.Chuah [9] claims that students who study at the department of engineering sciences must have analytical and logical thinking abilities so that they can make rational decisions on the foundations of engineering.Fatin [10] states that the importance of analytical thinking in engineering education where students are expected to make an inference when comparing and reason when taking risks and making decisions.Robbins [11] claims that analytical thinking skill is necessary when solving out the facts and both analytical and logical thinking skills are necessary when solving problems.
Consequently, how we see education has taken a new form through the ideas related to the development of analytical and logical thinking skills.Through the computer and the Internet usage, a new generation growing up with digital games which is defined as Ggeneration (children who were born after 2000) [12].We cannot set apart this generation of today, which is called information age, from computer and the technology, which comes along with it.People of this generation are continuously using the Internet and prefer to be connected to the social networks.They use game elements and gaming principles in any of their activities constantly without even realizing it.The research shows that this generation plays games for more than 10.000 hours a day in average [13].Prensky [14] describes this new generation who was born and is growing up with the technology age as "digital natives".Students now learn by playing willfully and fondly in an environment that computer games provide [15].
Hou [20] suggests that games such as Farmville are played in order to have fun, relax and steer away from stress in daily life.Games, which are not designed to enjoy, first are called serious games.Topics such as education, trade, health, and social awareness are within the serious games.Bogost [21] discussed the existence of persuasive games.He claimed that games, which have a high level of persuasive ability, are video games and video games are games, which has already changed the structure of sociocultural systems with its potential to provide social change.
One of the most important concepts for using gaming technologies in education is called gamification.The quality of software designed using gamification largely affects teaching.While well-developed gamified applications should have a positive effect on individuals' success, badly developed software can have negative effects on individuals.Dominguez et al. [16] aimed at using the video games by transferring the good sides of them into environments by using them to train individuals.
Gamification is a different term from serious and persuasive games.While game components are used in non-game environments in gamification, there is a complete game platform in serious and persuasive games.This is a key difference between game and the term gamification.Especially, in recent years, gamification has started to be used in many fields of research such as software, engineering, business and medicine.
Zickerman and Cunningham [17] define the gamification as "the usage of thinking style in the game and the game rules to draw the users" attention and solve problems.Deterding et al [18] use the term gamification to refer to the usage of game design in contexts in which there is no complete game.In this regard, the design must be clear first so that gamification could be implemented.And if this design could be used in contexts or environments, where no real game exists, gamification process can be applied.
In this context, the game design to be carried out is very important.Bunchball [19] put in order the outlines of gamification components as points, badges, levels, experience and leaderboards.Points can be described as a prize given as a result of success achieved during the game.Badges are prizes given for the general success in the game.For instance, individuals who achieve different successes are also given rating badges.The applications that were using the game components effectively reached a big target audience.
Today smartphones and tablets not only keep up with desktop computers but they also have many benefits [22].The usage of education and productivity activities through smartphones, which G-generation prefers to carry them, improves in addition to portability, ease of use and ergonomics every day.
This study seeks to address the following two questions, which should concern both computer engineering students and software practitioners: 1-Do prospective students have substantial skills to study computer engineering?Can we reveal the potential success that students could achieve as a software engineer when they graduate?2-Can gamification techniques be a useful asset to improve participant's engagement?How effective is gamification for improving motivation?
Based on a questionnaire administered by the primary researcher, this research provides a mobile framework for the exploration of logical and analytical thinking skills of individuals.To this end, a self-assessment tool is designed with the help of a set of game elements so that questions could reach the Ggeneration more easily.Consequently, the goal is to improve the motivation and enthusiasm of these participants.To construct an efficient design framework, experts were consulted through a Delphi study.This way, a software design, which was fit for the purpose, would be prepared in an effective way.
At first, the proposed approach is tested on individuals were experienced in software development.Based on the acquired results from practitioners, a threshold value is formed which was taken as a reference value for evaluating the results of the students who participated in this study may obtain information about their skill level.
The present study makes several noteworthy contributions to software engineering body of knowledge.Firstly, there is no tool that is developed to assess software practitioners' thinking skills in Turkey.This study initiates a formal attempt to explore such skills and their relationship with novice software practitioners.Secondly, to the best of our knowledge, for the first time, these assessments were conducted in Turkish software industry, which offers some insights regarding status of these skills for software practitioners.Thirdly, the requirement analysis of the mobile application was conducted using a novel approach.The functional requirements of developed tool were investigated using Delphi method.Finally, participants' engagement is provided by basic gamification techniques.However, as there is no previous study conducted about analytical and logical thinking of software practitioners, researchers could not be able to compare these results with other works.
This paper has been divided into four parts.The first section presents an overview of the study.The second part details the methodology that was used in this study.The third section presents the findings of the research.In addition, it analyses the results of interviews and focus group discussions undertaken during the study.Finally, fourth section discusses an overview of the findings of the study and offers concluding remarks.

Material and Method
Gamification is a technique to engage participants by using a set of game elements that likely to result extrinsic and intrinsic motivation.This study employed the PBL (i.e. points, badges and leaderboards) triad, which promotes reward-seeking behavior.It is considered as the three most effective game elements in gamification.Points are frequently used for understanding how well a player doing in a game by keeping the score.It can give continuous feedback to a participant.Most importantly, it can reveal the wining states, which can be connected to reward structures and provide data to the gamification designers.Badges are the representations of achievements that should signal the importance in form of a social display.Leaderboards are a kind of a social ranking and feedback mechanism by comparing participants with each other.In addition, an avatar is a frequently used game element to improve motivation and enhance engagement.It helps a participant to represent himself as a character in the application environment.

Test material
The current study involves an investigation of the effects of analytical and logical thinking skills on student life and business success of computer engineers and software developers.To accomplish this aim, SOHAT and TOLT, which have been used before and were proved to be valid, are used.
Umay and Arıol [23] developed "The Scale of holistic and analytical thinking" (SOHAT) to specify the level of students' analytical thinking.Taking into account the features of holistic and analytical thinking styles, 5 items that were thought to express the reflections on problem solving performance of these styles were developed for SOHAT.18 people were consulted as specialist opinion for the validity studies of the developed scale.The reliability coefficient of the scale was calculated as 0.78.
Tobin and Capie [24] developed the Logical Thinking Test (TOLT), which consist of ten.These are used to measure 5 reasoning forms: controlling variables, combinatorial reasoning, probabilistic reasoning, correlational reasoning and proportional reasoning.Analysis of the data revealed a high level of test validity and reliability (coefficient alpha= .85)

Research process
During this study, qualitative observation method of two stages was used in order to provide the functionality and quality of the software to be designed and put into use in certain criteria.
In order to reach the goals of study, the qualitative observation methods to be used were focus group and Delphi method.These methods were chosen to utilize a group of experts, and reach first-hand information.
We have three main titles, which were used in the two methods to be carried out: planning, data collection and analysis.In the first title, the validation of the titles in Table 1 was discussed as well as a focus group created by experts.The results obtained were listed.Delphi method was used with a group of different experts for the second method.Here, the goal was to prioritize the titles obtained at the end of the elimination and change the contents, which were possible to change.The application phase of this plan is as follows (see Figure 1).

Focus group
Focus Group is a study group of people picked out of experts on a specific topic.In order to identify the opinions and attitudes of those who join the group, the person conducting the research asks the participants predetermined questions.In the light of data obtained, in what way the planned study will be carried out is determined [26].The main purpose of using focus group is to develop a richer understanding and a point of view by getting away individual opinions blended with hundreds of ideas about something [27].The steps of process and the results will be discussed in three main titles.The right to come back to the previous question and change the answer should be given.

Time Management
An award should be given according to the speed of solving the problems.

Repeatability
The test should be used by the same person again.9

Real-time Feedback
The answer of each question should be given simultaneously.10 Calibration The calibration of the questions should be carried out according to the threshold value obtained earlier.

Presentation of the results
At the end of the test, the results should be ordered according to the leaderboard.
12 Avatar Avatars should be assigned to individuals according to their success level.

Planning
Two gamification experts and one android developer expert were invited by e-mail in order to create the focus group (see Table 2).The average age of the experts was 35.All the experts invited accepted the invitation to make a contribution to the validation of the software to be conducted.

Data collection
The session was started after all the experts gathered on skype calling and it continued for about 2 hours.Notes required were taken during the session.
All the titles and explanations were examined one by one.The facilitator explained the titles and the purposes of their formation and broached to the experts.An acceptance and a rejection list were made after getting the expert opinions.At the end of the session, both lists prepared by the facilitator were presented to the experts and it was verified that they were prepared according to their ideas.

Result and analysis
The two lists prepared by the focus group are as follows: -Validated List: 1, 2, 3, 5, 9, 10, 11, and 12.They are showed in Table 1.
33.3% of the topics was rejected by the experts during the session and the reasons for their rejection were explained in Table 3.The titles (topics) accepted were transferred to the next section and Delphi method was applied.

Delphi method
Delphi technique, which is a qualitative research method, is used to make a guess about the future by consulting experts.In 1946, Delphi method was used to identify security flaws especially in military by RAND Corporation in USA and it was used in the literature [28].Besides military, Delphi technique was used in IT [29,30] and education [31,32].
The purpose of Delphi study is to reach the most accurate consensus in order to gather a panel of experts and ensure controlled feedback by conducting questionnaires [33].The aim is to build consensus for the experts with different point of views for solving problems before they face the Delphi method.Delphi technique is building a structure where a group of individuals can communicate effectively in order to overcome complex problems [34].

Planning
Planning consists of the first and second steps which are 'identify' and 'select'.
The purpose of using Delphi method is to verify the titles accepted as accurate and applicable by focus group.5 gamification experts and 5 software developer experts were invited by e-mail apart from the experts who were invited to the focus group.But only 4 gamification experts and 3 software developer experts accepted the invitation.The Delphi method applied consists of 2 rounds.In the first round, the titles accepted by focus group were asked to be prioritized starting with the items which did not need to change.In the second round, the changes that must be made in the related items and the reasons for these changes were asked to be identified.Table 4 shows the experts' job, age and education.

Data collection
In data collection, the titles accepted by focus group were sent to the experts who accepted to participate in the study.The experts were asked to prioritize the titles that were sent in the first round from high to low order.Next, they were asked to specify the changes in the items, which needed to change from low to high order.The common titles were corrected and presented to the experts for approval.

Result and analysis
This title consists of the 4 th , 5 th , 6 th and 7 th steps of Delphi method where the analysis of the opinions, the change of the opinions, reasoned comments on these changes and the use of the results obtained from the experts are involved.
The results of the first round are presented in Table 5.Each of the results sent by experts is prioritized.
Kendall's Coefficient of Concordance (W) can be considered as a measure (i.e.assessment) of the agreement between a group of experts who have rank ordered a set of entities, which is a value between zero and one.The high level of agreement is shown above 0.7, while a moderate agreement is valued around 0.5.The values below 0.3 are accepted as weak agreement.Our results indicate that W = 0.575 which means there is a moderate agreement between the opinion of the consulted experts.
From the data in Table 5 we can see that the 11.title was chosen for the first position by 3 experts.The 12. title was chosen for the second position by 2 experts.The ninth title was chosen for the third position.The tenth title was chosen for the fourth position by 2 experts.The fifth title was chosen for the fifth position by 3 experts.The third title was chosen for the sixth position by 3 experts.The second title was chosen for the seventh position by 3 experts.Finally, the first title was chosen for the eighth position by 4 experts.When ordering the titles, the top rated title was chosen in each position.If the vote rate of the same title in two positions had been the same, only that title would have been sent to the experts to vote.Based upon the results, the new ordering is shown in Table 6.In the second round, information was obtained about the validity and stability of the content of the titles by experts.The experts were asked to identify which contents they needed to change, how they must be changed as well as the reasons for these changes.The contents and the details that experts asked to change are presented in Table 7.
The table obtained was sent to the experts for voting again.The experts sent the changes that their colleagues requested to be made to the method administrator by marking the ones, which they thought were positive again.The administrator examined the tables, which were marked and then determined the changes selected by a large majority and organized the application framework for its final form (Table 8).
By the end of the study, the application framework, which was arranged in the light of expert opinions and verified by experts, took its final form.The applicability, reliability, validity and gamification (gaming) of the mobile application to be designed were achieved in the highest motivation level.

A sample interface of the application.
After the users solve all of the survey questions in the application, the answers to the questions are sent to the database in our online system and evaluated here.
Based upon the results of evaluation, the success and the badge that the users reach are determined.Additionally, the success of each user is compared with the other users' success and then their ranking is identified.As shown in Figure 2, the ranking and the badge are presented to the user.Table 7.The contents that are asked to change and the reasons for these changes.

Expert 1 Real-time Feedback
The answers should be given at regular intervals.

Expert 2 Calibration
The average success of the ones taking the test should be used for the calibration results as well.
"The mobile application will not be enough for the target group."The mobile application to be designed is enough for the target group.
3 Motivation Gamification methods should be used in order to motivate the individuals taking the test.9 Real time Feedback As the goal is approached, the process will continue.

Calibration
The course success of individuals will be taken into consideration in addition to threshold value for the calibration results.11 Presentation of the results At the end of the test, the results should be ordered as per leaderboard.12 Avatar Badges should be assigned to the individuals depending on their success level.

Results
The tests, which aimed to identify the success of these abilities on professional life, are applied to the software developers and control group was created.Furthermore, the success rates of the test applied to students of computer engineering were compared to the scores in class and the impact of these abilities on the success of school subjects is testing.The details of the studies were described respectively and shown in tables.Then the results were discussed.

Control group
The aim was to identify the business success of students who studied at the department of computer programming, after they graduated.To this end, the analytical and logical thinking abilities of 20 computer programmers, who began to work, were measured.
 SOHAT and  TOLT surveys were conducted.
5 were the lowest point and 15 were the highest point according to the values of SOHAT results determined by Umay [23].It was also determined that the ones who got 5, 6 and 7 points had a dominant ability of analytical thinking and the ones who got 12, 13, 14, 15 points had a dominant ability of integrative thinking.
According to the score intervals of evaluation of TOLT results, the ones who had a low, average and high level of logical thinking abilities were identified.According to this, the ones who are in 0-3 scores have a low ability of logical thinking, the ones who are in 4-6 score intervals have an average ability of logical thinking, and the ones who are in 7-10 score intervals have a high level ability of logical thinking [35].
The scores that the computer engineers received from the surveys are presented in Table 9.
The distributions of the scores that the computer engineers received are shown in Table 10 and the graphical display of percentage distributions is shown in Figure 3 and the success distribution of test results is presented in Figure 4.
At the end of the SOHAT, the neutral ones were left out of assessment [23].When this section, which constitutes 67% of results, is ruled out, the remaining part is 33%.The individuals who constitute 29% of 33% are individuals with analytical thinking ability.The findings of TOLT percentage distributions reveal that individuals who have a high level of logical thinking ability with 50% are the largest mass.The findings also show that individuals with 43% have an average level of logical thinking ability and finally, individuals with 7% have a low level of logical thinking ability.
The results of the tests applied to the control group show that the individuals, who have a dominant structure of analytical and logical thinking according to the percentage distribution and success score distributions, became an engineer and continue their career successfully.

Experimental group
The same tests applied to the control group were exercised in order to determine the success that computer engineering students as experimental group will get in business life after they graduate and the results were analyzed.As an experimental group, 23 students of computer engineering were got involved.
The scores that the students got from the surveys are presented in Table 11 and the graphical representation of the scores is shown in Figure 5.The percentage distribution of the scores by individuals is presented in Table 12 and the graphical display of percentage distribution is given in Figure 6.
The results of the surveys and the graphics demonstrate that 65% of the results is the neutral part, which is eliminated.What is left is 35%.The individuals who constitute 30% of the remaining part with 35% are individuals with analytical thinking ability.Comparing the results of control group and experimental group, it can be seen that the 12 students who constitute experimental group with 52% can achieve success in business life in future.Also, it can be seen that the group of 8 students who has an average level of logical thinking ability with 35% need to work a bit more and improve themselves before they begin to work.The results also reveal that the remaining 13% section must work really hard in order to begin to work and reach success or they can prefer other fields of computer engineering apart from software development.

Validation interviews
Interviews were conducted with software developer experts in order to check the validity of our functionality and design approach.Five experts were contacted.First, the design was applied to the experts and then they were asked to give information about the process of the design, possible problems, and the effects of the design on individuals and what to do in future.The answers of the experts can be listed as follows: Expert 1: "The [gamified] design is colorful and pleasant from the very beginning.It creates the desire to solve the problems without getting bored.I did not encounter any problem as to the way it works during the processing time and it really works uncomplicatedly.It is useful that there is an explanation before each test.The 3 rd level badge gained at the end of the application can make the user feel the joy of success and pride.I think that it will create a competitive environment for the students using the application and thus, their success levels will increase."the more the number of students using the application increases, the more determination in the order will be.The application will lead not only to university students but it will also lead to students preparing for university." Expert 5: "I think that the application and the questions related to thinking abilities are fit for purpose.The application is simple and useful.In order to see the reward, I would get and my grading at the end of the test, I solved the questions curiously by having fun.I found that the evaluation was accurate and determined at the end of the application and I really believe that it should be used to lead to individuals.To improve the application, the number of target audience can be increased by adding relevant questions to fields of engineering."

Threats to validity
Potential factors, which can affect the results of the study in a negative way and reduce the validity and reliability of them, are defined as threats to validity [36].Because the coefficient of validation was tested by the researchers who designed the TOLT and SOHAT assessments, there was no need to evaluate the construct validity.
However, there might be some threats, which likely to affect the results negatively in our study as well.These threats can be listed as follows:  The student may create a fake email account and have the chance to use the application again.
 The student may find out about the questions from his/her friend and see the questions while someone else answering them and make a guess about the results before.
 The student may have the chance to reach the questions and answers before as the survey questions have been already published in many surveys.
Although we are very careful about the threats that might affect our study negatively and take precautions against them, the results of the study may be adversely affected for the reasons above.

Discussions and Conclusions
The aim of this study is to develop a gamified instrument to assess the analytical and logical thinking skills of computer engineering students, and to explore the potential success of students' as software practitioners when they start working at the software industry.To this end, there has been a necessity to identify the common features between individuals who finished their engineering education and began to work and students from the department of engineering or student applicants.
As a result of the study, different styles of thinking were examined for representing the common characteristics of the individuals.The results showed that many of the researches conducted on engineering and sciences focused on logical and analytical thinking abilities.TOLT and SOHAT were chosen for the study due to the fact that these tests are commonly used in different fields, were proved to be reliable and are easy to implement.In order for the individuals to solve this test, a set of game elements, which were developed to increase participant's motivation, was designed.Consequently, a competitive environment that provides instant results was built.Also, smart phone applications were designed to make the transportability easier.The importance of thinking skills and the contributions of mobile learning when implementing them were described and our research methodology was explained.It was explained why Delphi study, which is a qualitative research technique, was used and the procedure of the study were mentioned.The structure of the application designed according to a focus group study and the Delphi method was used to negotiate a group of experts.Finally, using the developed product, the results of SOHAT and TOLT, which were applied to the software developers and the students, were analyzed.The results indicate that more than 50% of computer engineering students and the ones who began to work are the ones who developed their analytical and logical thinking abilities.The findings reveal that these abilities of individuals have a strong effect on the success of being a software developer.Then SOHAT and TOLT were applied to the university students and similar results were found.When comparing the two groups, it can be seen that there is no obstacle that can stand in the way of half of the students being a successful software developer.Additionally, this test may be helpful for the student group preparing for university about whether they should study at the department of computer engineering or not.In addition to the test results, senior software developers were interviewed in order to strengthen the functionality of the application designed and the methodology used.
The remarks obtained from the study can be summarized as follows:  The platform creates an opportunity for students to assess themselves.In particular, individuals who are worried for their future success.Gamification can help them to overcome their concerns in a fun way.
 Based on the game elements, their confidence will grow thanks to the badges given as a reward and they will focus better not to decrease their success.
 The proposed application promotes the students to observe their skill levels.
 The gamification creates an interactive environment especially for participants to inform about the other participants.
 University applicants might benefit from the application.They would see the ordering between graduate engineers and students of engineering and thus, they could make a better choice for a department.
The method used and the software designed achieved their purpose and met the experts' expectations.By adding additional options, the design can be developed.Thus, people who will decide to study computer engineering can be sure of their decision.
First of all, what we are planning to do is to improve the gamification techniques in the design, which consist of gamification platform so that the tests could be more enjoyable and more motivating.Furthermore, we are trying to make it possible for this design, which can only be accessed from mobile platforms, to reach from a web-based platform so that it will reach larger audiences as well.
Finally, a preliminary prototype, which was developed can be used not only in computer engineering but also be beneficial to use in other domains that needs skills of analytical and logical thinking skills.When it comes to engineering, mathematical analysis cannot be separated from each other.Based upon the foundations of engineering concepts, questions, which contain other types of questions, can be formed and added to the tool so that a wide range of engineering students could benefit from such a platform as well.

Figure 2 :
Figure 2: A snapshot from the mobile application interface.

Figure 3 .
Figure 3. Percentage Distribution of Control Group Success.

Figure 4 .
Figure 4. Success Distribution of Control Group.

Figure 6 .
Figure 6.The Percentage Distribution of Experimental Group Success.

Table 1 .
Implementation Framework for functional requirements.

Table 2 .
Expert Reviewers' Information for Focus Group.

Table 3 .
Reasons for rejection.

Table 4 .
Expert Reviewers' Information for Delphi Study.

Table 5 .
Round 1 classification of validated for each expert.

Table 6 .
Final classification of validated.

Table 9 .
Control group test results.

Table 10 .
The Distribution of Control Group Test Results by Significance Level.Success Distribution of Experimental Group.

Table 11 .
Test Results of Experimental Group.

Table 12 .
The Distribution of Experimental Group Test Results by significance levels.