A Validity and Reliability Study of the Engineering and Engineering Education Attitude Scale (EEAS)

In the literature it was not possible to find a proven valid and reliable scale to measure students’ attitudes towards the engineering profession and engineering education. Such scale is the objective of the present research. The sample group is composed of 650 students for the first application and 113 students for the second. In order to detect the validity of the scale, exploratory and confirmatory factor analyses, item factor total correlations, corrected correlations and item discriminations were conducted. In order to assess the reliability of the scale, the level of internal consistency and the stability levels were calculated. EEAS is a five-point Likert-type scale and includes 17 items with two factors. The analyses provided evidence that EEAS is a valid and reliable scale that can be assuredly used to identify students’ attitudes towards the engineering profession and the education they receive. Keywords—Engineering education, Engineering, Attitude, Scale Development


Introduction
It is quite reasonable to claim that there are considerable factors effective on learning. However some factors, compared to the rest, leave a further critical effect on learning. These critical factors are naturally the ones that occupy a wider space of discussion. Among these, attitude is one of the significant factors. Attitude is briefly defined as negative or positive emotions of an individual towards performing any given action [1]. Relevant to the literature, there are numerous studies validating that students' attitudes towards school, profession, particular lesson or teacher are directly effective on a vast number of psychological variables, the foremost of which is academic achievement [1][2][3][4]. But there is limited edition of studies on engineering education in particular; nevertheless a closer investigation of studies related to the teaching of miscellaneous subjects show that one of the critical factors impinging upon students' academic achievement is attitude [5][6][7].
It is safe to argue that in the formation of attitudes in an individual's life, school and class environment play a vital role. The attitudes that people develop during their education life towards certain variables, such as teacher, lesson, school, etc., may leave effects on their present learning life and in the future profession [8]. Knowing the fact that people can self-realize themselves through the profession they select, the harmony between one's own profession and his/her interests and skills, or, in other words, the positive attitude towards one's own profession, can provide positive outcomes such as success, productivity and performance rise, not only in the educational life they have before starting work, but also in their business life after entering into work.
It should be acknowledged that in the present age we heavily rely on scientific and technological innovations in daily life and leading innovations are witnessed in economy and national security. Engineering is the key component of technological society and innovation [9]. Within that context, in almost all countries, particularly in the U.S., the authorities feel the urge to revise their education system on the basis of the quartet comprising Science, Technology, Engineering and Mathematics (STEM) [9,10]. STEM can be described as a complementary educational approach generally emphasizing the co-acquisition of the kind of skills constituting the foundational pillar of technology education [11]. On that account, it can be claimed that the foundation of engineering education is also formed by STEM [12]. In response to the rising demand for technology, it is feasible to assert that a more strategic importance of engineering can lead to a better quality engineering education and continuous popularity of accreditation issues. Within that framework, there is a rising trend in curriculum development and alternative learning-teaching activities. STEM is one of the vital products of this process. Engineering faculty graduates of the present day are commissioned to solve the problems of a future world that faces a rapidly increasing and more critical than ever set of challenges [13]. Therefore, they are naturally expected to possess a number of integrated skills. Correspondingly, it can be stated that in the society there is a rising demand for engineers. It is, however, a bitter reality that those students are, due to psychological, economical or other factors, likely to perceive engineering education as a step to move to other professions to attain a higher social status or higher income [13]. At this point, the gravity of the positive attitude of students towards the engineering profession and engineering education is emphasized one more time [13].
In research related to the problems witnessed in engineering education, the prominent issues are the failure of some students to attach required importance to basic sciences, a nonchalant attitude towards lessons and a lack of desire to further improve their knowledge outside class hours [14]. The presence of such and similar student-induced problems in engineering education may be explained with the negative attitude of students towards engineering education as well as the profession. By the same token, the identification of prospective engineers' attitudes towards their education and the profession bears vital significance in terms of their undergraduate education process and their professional performance in the future. Despite this necessity, a closer inspection of relevant literature reveals that there is lack of sufficient evidence on the attitudes of prospective engineers directed towards engineering education. Furthermore, in literature it was not possible to find a proven valid and reliable scale to measure students' and prospective engineers' attitudes towards the engineering profession and engineering education. Given the impact of attitude on academic processes, it is reasonable to claim that such a scale is required.
Furthermore, this scale can render great contributions to relevant literature and is therefore the objective of the present research.

Study Group
For the first application, the study group of the present research consisted of 650 prospective engineering students (155 female and 495 male) studying in different Engineering Departments or Technology Faculties in Amasya, Abant İzzet Baysal, Dumlupınar, Düzce, Marmara and Selçuk Universities in Turkey. In the second application, in order to conduct the confirmatory factor analysis and test-retest method, the same application was performed among 113 students (25 female and 88 male) in Amasya University, Faculty of Technology. In the first application there were 29 items in the item pool, whilst in the second application 17 items were considered. In scale studies, the popular suggestion is for the number of participants to reach at least 5 times the number of items in the pool. In line with this suggestion, 650 participants were involved for the first application and for the second application 113 participants were deemed to be sufficient. In Table 1 the distribution of the study group with respect to department and gender is summarized.

Scale Development Process
In the process of scale development, the first step has been literature review and certain items in Attitude Scale towards Professional Education previously developed by Kalkan [15] have been adapted to the scale. At the same time, the researchers prepared the relevant items for the purpose of measuring attitude. Additionally, fifteen sophomore students attending Amasya University, Faculty of Technology, Department of Mechanical and Electrical-Electronic Engineering were interviewed about their feelings towards the engineering profession and asked to write down, in an open-ended format, their opinions about the ongoing engineering education. After analyzing the obtained texts, the sentences were transformed into scale items. At the onset, two factors were envisaged within the scope of scale: attitude towards the engineering profession and engineering education. On the basis of these factors, a pool of 29 items was created with the noted items. The item pool created via this method was examined by one education programmer, one expert in guidance and psychological counseling and one computer engineer to check overlapping items and content validity. Next, a linguist was consulted to correct any potential abstruse expressions or wrong statements in the pool, if any. The item pool was prepared as a draft form and pilot practice was launched.
In this pool there were 17 negative and 12 positive statements. Five-grade choices were placed opposite the items to measure attitude levels stated in the items. These choices were sequenced and graded as "(1) I totally disagree", "(2) I disagree", (3) I partially agree", "(4) I agree" and "(5) I totally agree".
The finalized form of scale was duplicated and conducted on participants. In order to check the statistical validity and reliability of the scale, collected data were loaded into the SPSS 15.00 and AMOS 16 programs. While loading into the programs, reverse coding was followed to upload the values related to negative statements.

Data Analysis
In order to establish the validity of the scale, the initial step has been to test the structural validity. To identify the structural validity, Kaiser-Meyer-Oklin (KMO) and Bartlett analyses were conducted to check the applicability of factor analysis [16]. Based on the obtained values, exploratory and confirmatory factor analyses were conducted on the data; the divisibility of the scale into factors was determined via principal components analysis and, by utilizing the Varimax orthogonal rotation technique, factor loads were examined. Factor analysis is used to detect if items in any given scale can be divided into fewer numbers of factors [17]. Provided that factor loads of scale items are above 0.30 and explain a minimum of 40% of general variance, the scale is adequate, as also recognized in behavioral sciences [18][19][20][21]. Scale form obtained via exploratory factor analysis was conducted among a new study group different from the participants in the first application and confirmatory factor analysis was administered to the collected data. Confirmatory factor analysis is based on the principle of treating and testing the relations between observable and unobservable variables (items and factors) as single hypotheses. In a different saying, confirmatory factor analysis is a structural equation model dealing with measurement models of the relations between latent variables and observed variables [16]. Every single factor is explained in terms of the relations with observable variables (items) [7,22]. In confirmatory factor analysis, the maximum likelihood technique was implemented. At the end of factor analysis, the distinctiveness power of the remaining items in the scale was checked via independent sampling t test and item-total correlations were checked via Pearson's r test to measure the validity quality of our scale. The existence of a correlation between the score received from each single item and the score obtained from the item factors is utilized as criterion in determining the level of serving to the general objective of each factor by each item in the scale [17]. The distinctiveness feature is recognized as one of the major evidences in determining the validity of any scale [18]. Another method to test the distinctiveness of a scale is, after sequencing the raw scores received from each item from the large to the small, to observe the differentiation between the lower 27% and the upper 27% groups.
To the end of identifying the reliability of the scale, stability tests were performed with internal consistency coefficients. In the designation of internal consistency level, Cronbach alpha reliability coefficient, correlation value between two equal halves, Spearman-Brown formula and Guttmann split-half reliability formula were harnessed. Reliability coefficients of 0.70 and higher point to the reliability of the overall scale [18,23]. The stability level of the scale was checked by detecting the correlation of results between two applications conducted in five-week intervals. As acknowledged, a reliable measurement tool is required to conduct stable measurements [17]. In addition to that, reliability is also linked to the stability, consistency and sensitivity features of the scale. Hence, these values identified as stability coefficients are recognized as evidences of the reliability of the scale [24]. Reliability coefficients standing for consistency degree increase towards 1.00 and decrease towards 0.00 [23]. Generally speaking, for correlation coefficients, the 0.00 -0.30 level indicates a low relation, 0.30 -0.70 indicates an average relation, and 0.70 -1.00 indicates a strong relation [18].

Findings on the Validity of Scale
Within the framework of the validity of the Engineering and Engineering Education Attitude Scale (EEAS), the structural validity, item-total correlations, corrected correlations and item distinctiveness were analyzed and the obtained findings are as presented hereinafter.
Structural Validity: Exploratory factor analysis results -To test the structural validity of EEAS, firstly KMO and Bartlett tests were applied to the data and the obtained values are KMO=0.823; Bartlett (χ 2 =674.733; df=136; p=0.000). Within the framework of the obtained values, it was construed that factor analysis on this 29 item scale was feasible. To determine whether the scale was single-dimensional, principal components analysis and, with respect to principal components, Varimax orthogonal rotation technique were employed. In this aspect, 9 items whose item load was below 0.30 and 3 items whose load was divided among different factors, corresponding to a sum of 12 items, were excluded from the scale and factor analysis was repeated on the remaining items. To ensure that content validity was not disrupted due to item exclusion, the created item pool was reexamined by the very same field experts. Subsequent to receiving the confirmation that the exclusion of 12 items had no adverse effect on content validity, it was then possible to implement the remaining analyses. Finally, there were 17 items left in the scale and the remaining items were collected under two factors. It is detected that unrotated factor loads of the 20 items in the scale are between 0.307 and 0.645. It was also seen that total variance of items and factors included in the scale explained 44.354% of the total variance. Table 2 shows the factor loads as well as the eigenvalues and the explained variance for each factor.
As demonstrated in Table 2, the two-factor structure envisaged in the beginning is confirmed by the exploratory factor analysis.
The "Attitude towards Engineering Profession" factor includes 9 items whose factor loads vary from 0.525 to 0.803. The eigenvalue of this factor in the overall scale is 5.812; its contribution level to general variance is 23.068%. The "Attitude towards Engineering Education" factor involves 8 items whose factor loads vary from 0.504 to 0.776. The eigenvalue of this factor in the overall scale is 1.728; its contribution level to general variance is 21.286%.
This finding is also demonstrated in a curved accumulation graph drawn with respect to eigenvalues (Fig. 1), in which it can be witnessed that there are sharply accelerated falls in the first two factors, which indicates that they contribute massively to the variance; on the other hand, the fall in the other factors tends to follow a horizontal course which indicates that their contribution to the variance is of an almost identical degree [18,19].

Item Factor Total and Corrected Correlations:
In this part, in order to test the level of service of each item to the general objective, correlations have been measured between the scores received from each item in the scale with respect to item-total correlation and corrected item correlation method and the scores received from the factors. The obtained item-factor correlation values and the corrected correlation values for each item are as seen in Table 3. As can be seen in Table 3, item test correlation coefficients vary for the first factor between 0.502 and 0,686; and between 0.530 and 0.680 for the second factor. Each single item is in a significant and positive relation with the overall factor (p<0.001). Also, as can be detected in Table 3, the corrected correlation coefficients with the factor of each item in the scale are, for the corrected first factor, between 0.330 and 0.571; and for the second factor between 0.380 and 0.553. It can thus be argued that every single item serves to the objective of its own factor.
Item Distinctiveness: The distinctiveness power of the items in the scale was measured. To that end the raw scores received from each item were sequenced from the large to the small. Next, upper and lower groups of 176 participants composing the lower 27% and upper 27% groups were determined. Independent groups t-test values were computed on the basis of total scores in the groups. Findings of the t values and level of significance of their distinctiveness power are shown in Table 4.  Table 4 shows that the first factor items vary between -10.059 and -17.020, with -36.657 for the factor sum; it is between -11.145 and -20.157 for the items in second factor and -32.867 for the factor sum. The t value of the overall scale was computed as -51.317. The level of each identified difference is significant (p<0.001). It can thus be argued that the distinctiveness level of the overall scale and also of each specific item in the scale are high.

Findings related to the Reliability of the Scale
For measuring the reliability of the scale, internal consistency and stability analyses were conducted on the data. The conducted procedures and obtained findings are as listed below.
Internal Consistency Levels: The reliability of the scale as a whole and with respect to factors was measured via Cronbach alpha reliability coefficients, correlation value between two equal halves, Spearman-Brown formula and Guttmann split-half reliability formula. The reliability analysis results with respect to each factor and overall scale are as summarized in Table 5. As demonstrated in Table 5, the correlation value between two equal halves is 0.667, Spearman Brown reliability coefficient is 0.800, Guttmann split-half value is 0.791, Cronbach's alpha reliability coefficient is 0.853. On the other hand, it is also seen that factors' equal half correlations are between 0.529 and 0.575, Spearman Brown values are between 0.692 and 0.730, Guttmann split-half values are between 0.685 and 0.726, Cronbach's alpha values are between 0.765 and 0.769. Accordingly, it is feasible to claim that the scale is able to meet consistent measurements since internal consistency coefficients of individual factors and also the overall scale are sufficiently high. Stability Level: The stability level of the scale was measured by employing the test-retest method. Four weeks later, the final version of the scale with 17 items was conducted on 41 students who had received the first application. The relation between the scores received at the end of both applications was analyzed in terms of both each single factor and overall scale and the obtained findings are summarized in Table 6. In Table 6, it is seen that correlation coefficients of each scale item obtained via the test-retest method vary between 0.697 and 0.783 and every single relation is significant and positive. It is also witnessed that correlation coefficients obtained via test-retest method for the scale factors are 0.727 and 0.731, correlation of the total score is 0.725 and every single relation is significant and positive. Hence, it is safe to argue that the stability level of the scale is extremely high.

Discussion
In the present research a scale has been developed to detect the attitudes of students towards the engineering profession and engineering education they are currently receiving. EEAS is a five-degree Likert type scale and consists of 17 items that can be grouped under two factors. At the onset, it was envisaged to form the scale with two principal factors and items were listed accordingly. One factor is "Attitude towards Engineering Profession" and the other one is "Attitude towards Engineering Education". As it is suggested that both factors are equally important in analyzing the academic achievement of engineering students, the scale has been developed within the framework of the two factors. A closer look at the general objective of modern engineering education reveals that it goes way beyond instilling certain technical knowledge to students or raising the kind of engineers capable of solving technical problems alone. It is actually geared at training engineers that hold a systematic perspective [14]. Thus, it can be argued that prospective engineers in universities are expected to exhibit certain skills that go beyond being fully equipped in the scientific and technologic domains. An individual's attitude towards his/her profession determines the way s/he acts in his/her work, or, in other words, it indicates this person's professional performance, which in turn determines the level of achievement [25]. It is considered that the attitude of people towards their education received during candidacy period is equally effective in their attitudes towards their profession, since a person can have the opportunity of learning and developing his/her future profession while s/he is receiving undergraduate education. From this perspective, it can be stated that the identification of prospective engineers' attitudes towards the engineering education process as well as the engineering profession is a closely intertwined subject concerning their academic and professional achievement.
To illustrate it better, validity was analyzed via two different methods with respect to factor analysis and distinctiveness feature. In order to designate to what extend each item in the scale could measure the features attempted to be measured via the factor it belonged to, item total correlations and corrected correlations were computed on the obtained data. The obtained values proved that each item and each factor in the scale significantly served the objective of measuring the features aimed to be identified with the overall scale. Additionally, t values relevant of the difference between upper 27% and lower 27% groups were examined to explore the level of their distinctiveness. It was concluded that, both the overall scale and each specific item of the scale, possessed high distinctiveness character, or, to put this in a different way, it was verified that each single item exhibited the required level of distinctiveness. Internal scale consistency coefficients were measured by applying two equal half correlations, Cronbach alpha, Spearman-Brown formula and Guttmann split-half reliability formula. Within the framework of computed values, it was ascertained that the scale was capable of conducting reliable measurements. In order to determine time-independent uniformity of the scale, the test-retest method was applied by utilizing data collected in five-week interval applications. The test-retest method was measured within the framework not only of each specific item, but also for the sub-factors of scale. It was thus concluded that each item in the scale and each factor could conduct stable measurements time-independently.
To conclude, it is reasonable to assert that EEAS is a valid and reliable scale that can be assuredly used to identify students' attitudes towards the engineering profession and the engineering education they receive. In the relevant literature, there is a lack of such a measurement tool. By the same token, it is considered that the subject matter of the present research can render significant contributions to the literature.