DEVELOPMENT OF A SCALE TO MEASURE FACULTY ATTITUDE TOWARDS OPEN EDUCATIONAL RESOURCES

This paper describes the entire methodology for the development of a scale to measure Attitude towards Open Educational Resources (ATOER). Traditionally, it is observed that some teachers are more willing to share their work than others, indicating the need to understand teachers’ psychological and behavioural determinants that influence use of OER. The paper presents the methodological rigour in the development of the 17 items two-factor scale that is valid and reliable to measure attitude towards OER. The psychometric properties of the scale include: Content Validity Ratio=0.9 and Cronbach α=0.897 with strong inter-items correlation. The two-factors attitude construct in the scale was also subjected to a good model fit using Structural Equation Modelling, which revealed a mediocre fit with 0.8 Root Mean Square Error Approximation value and the chi-square to degree of freedom ratio below 3. We also discuss the significance of the scale and how to use it with other variables effectively in different contexts to help develop appropriate strategies for promoting the use of OER in educational institutions.


Introduction
Open educational resources (OER) have emerged as one of the most useful teaching-learning practices in educational arena (Dhanarajan & Porter, 2013;Glennie, Harley, Butcher & van Wyk, 2012). They have been used to reduce time to develop courses and facilitate sharing of knowledge. To teachers and students, OER provide access to global content that can be localized without restrictions and create inclusive learning communities (Butcher, 2011). Mostly, OER are prepared by teachers for different learners in a specific context. Therefore, place of teachers and their attitude towards open education-to provide those conditions that would engage their learners as active participants-becomes essential.
Open Praxis, vol. 8 issue 1, January-March 2016, pp. 55-69 However, several research studies reported that learning was tempered by teachers' lack of expertise in OER. Petrides, Jimes, Middleton-Detzner, Walling and Weiss (2011) reported that faculty with lower comfort levels in using online technology uses open textbooks in more traditional ways; which hampers independent learning among students. But, with arrival of digital technologies, it has become easier for teachers to share their work not only with their students, but it has also offered opportunity to share their work globally. More specifically, this development encourages them to further develop, practice and model new behaviours with their students. Therefore, there is a need to understand teachers' psychological and behavioural determinants that may influence better use of OER.
On this premise, our study involves understanding why some teachers share educational resources and others do not. In order to investigate this, we examine the OER perception (use and contribution) by teachers in universities as a combined intertwined psychological constructs of teacher's attitude, motivations, their perception of quality and barriers. While the research is in progress, this paper describes development of a scale to measure Attitude towards Open Educational Resources (ATOER) within the framework of a research project in the global south to explore the use of OER and evidence of impact of OER. Thus, it discusses various phases of development and validation of a scale to measure faculty attitude towards OER and presents the findings of Content Validity Ratio (CVR), factor analysis and reliability co-efficient to report the psychometric properties of the scale. It also expands on a previous paper presented on validation of the scale at the 28th Annual Conference of the Asian Association of Open Universities (AAOU) from 28-31 October 2014, at the Open University of Hong Kong (Sharma, Mishra & Thakur, 2014).

Review of Related Literature
Review of literature is divided on the basis of three sets of constructs extracted from various studies: Awareness of OER, Sharing of Resources, and Adoption and Use of OER.

Awareness of OER
First set of studies (Mtebe & Raisamo, 2014;Jameela, 2014;Karunanayaka, 2012) have assessed teacher's attitudes through understanding of their 'Awareness' of OER. These studies reveal that many teachers are not even aware of the concept and meaning of OER. Some of the teachers who are aware of the concept are not clear about copyright issues (Karunanayaka, 2012;Jameela, 2014). Nonetheless, there are teachers who have both knowledge and concept of OER and copyrights, yet they are not able to share or use their resources due to lack of technology skills (Mtebe & Raisamo, 2014).

Sharing of Resources
Second set of studies (Wang & Noe, 2010;Wild, 2011;Rolfe, 2012;Tuomi, 2013) have identified that the OER movement is primarily based on individual's desire to borrow and 'share resources'. Belief in open education, economic reasons and as a reputation enhancer-both for institution and individual-emerged as strong communal drivers for sharing resources (Rolfe, 2012). Additionally, there are several motives behind sharing behaviour, such as altruism, prestige and reciprocity, which may motivate teachers to share (Wang, & Noe, 2010). In addition, OER sharing also facilitates selfdirected learning (Tuomi, 2013). A sense of belonging, shared purpose, and empowerment are the greatest drivers for sharing resources (Wild, 2011

Adaptation and Use of OER
A third set of studies (Pegler, 2012;Hussain, 2013;Borthwick & Gallagher-Brett, 2014) investigated factors associated with 'Adaptation and use of OER' determining teachers' attitude for engaging in OER. Free availability and reusability of OER, their reduced cost and ease of use are major reasons for teachers to adopt and use OER (Borthwick & Gallagher-Brett, 2014). In addition, amicable technology, teachers' competencies, and their ICT skills also determine grounds for adopting and using OER (Hussain, 2013). For reusing OER, positive environment and availability of appropriate open licensed resources were found to be major factors (Pegler, 2012).

Rationale of the Study
Developing a measurement scale that is valid and reliable is always challenging. Several scholars argue that effective measurement is an underpinning of research (DeVellis, 2003;Netemeyer, Bearden & Sharma, 2003). Besides that, reliable and valid measures contribute to the legitimacy and development of a research field (Reynolds, 2010). Also, empirical articles that use rigorous methodological procedures, besides being firmly grounded in theory, receive more citations (Colquitt & Zapata-Phelan, 2007).
Research in OER field is quite recent and is not common due to lack of awareness, funds to support researches and other contextual dynamics. There is also a dearth of empirical research that follows sound methodological approaches. One Indian study by Venkaiah (2007) examined attitude and perception of distance teachers towards OER using a scale that was not subjected to psychometric validation. Researchers on OER have yet to adopt rigour in conduct of empirical studies, as in other fields of education. It could be due to its emerging nature or being rooted within Educational Technology, Information Communication Technology (ICT) and e-learning rather than as an independent field.
The motivation for this research springs from gaps in earlier studies related to OER. Whatever available research on attitude towards OER, they do not try to investigate underlying constructs. Content domain specification and item pool generation are not explained in detail. While much importance has been given to questionnaires and interview schedules, very few used scaling techniques to measure attitudes. Moreover, relevant research findings were not always utilized for constructing sound scales to measure faculty attitude towards OER.
Building on the methodological inadequacies of previous works, the current research aims to construct a rating scale called Attitude towards Open Educational Resources (ATOER) that can precisely identify positive and negative pre-dispositions to the concept and practices of OER amongst teachers. Literature review provided a basis for developing three major constructs for ATOER scale -awareness, sharing of resources, and adoption and use of OER.
The study contributes towards the practice of rigorous scale development in researching OER, and describes critical steps in scale development procedure.

Methodology
This section outlines the steps for validity, reliability and optimisation of ATOER scale undertaken in this study. The methodologies used are elaborated below for each step:

Domain Identification and Item Generation
Generation of items is the most important element of establishing sound measures (Hinkin, 1995). In the process of developing ATOER scale, initially 65 statements were pooled from review of Open Praxis, vol. 8 issue 1, January-March 2016, pp. 55-69 literature and classified into three main themes: Awareness, Sharing of resources and Adoption and use of OER. Afterwards, to avoid duplication, and have clarity, only 26 statements were selected through sorting process based on rigorous discussions within the internal research team. These 26 statements were subjected to content validity by the research team. A pool of 30 experts was drawn from the research literature and various projects, such as WikiEducator and the Research on OER for Development (ROER4D) group.

Content Expert Validation
This study uses Content Validity Ratio (CVR) proposed by Lawshe (1975) to identify valid statements. This was accomplished in three stages: At first stage, only 30 experts were selected to express opinion on suitability of the identified 26 statements to measure attitude towards OER. They were asked to rate the statements in a threepoint scale (1= Not necessary, 2= Useful, but not essential, and 3= Essential). We used an online survey tool to collect data, and experts were also given a brief about context of the research. CVR was calculated as described by Lawshe (1975) to assess the content validity.
Followed by first stage, CVR was re-calculated combining both 'Essential' and 'Useful, but not necessary' ratings to give a combine value of CVR E+U at Second stage. This is a modified CVR approach (Kawachi, 2014).
At third stage, ATOER scale was further revised by adding more clarifying items. Language of the scale was further simplified, and it had 34 items. At this stage, we also separated items of the three constructs and sent the scale to the 30 experts, which resulted in only four additional responses.
We conducted another round of analysis, as the number of response in stage 3 was less. At this stage the average value of CVR E+U of second and third stage for all the items was calculated, and 8 items (item no. 2, 3, 13, 20, 27, 28, 29 and 34, from the third stage) were omitted owing to their low CVR E+U value. A final valid scale with 26 items was thus finalized for the next level of tests.

Administration of the Items to a Development Sample
The scale with 26 items was distributed online as well as in four face-to-face workshops on OER conducted in four different Indian universities. About 30 teachers attended each of the workshops, and we sent the online survey to about 150 OER practitioners on the WikiEducator India list. Tinsley and Tinsley (1987) suggest a ratio of 5 to 10 subjects per item, i.e. up to a sample size of about 300 for factor analysis. Thus, distribution of the questionnaire containing 26 items to a sample size of 270 was considered satisfactory, and a large sample would eliminate subject variance (DeVellis, 2003) for scale development. However, only 117 (43%) usable responses were received. Though this was considered as a limitation at this stage, the analysis of the responses found that this return rate was adequate for this instrument.
Detailed analysis of the psychometric properties of the scale for validity and reliability, including factor analysis, are described in the next section.

Validity of Items in the Scale
In order to examine the validity of ATOER scale, Content Validity Ratio (CVR) was calculated in four stages. Findings and analysis of each stage are discussed below: First Stage: A total of 19 experts out of 30 responded. However, only 15 responses were found to be complete. On the basis of the data, CVR was calculated to be -0.18, which is very less than critical value of 0.49 at p<0.05 level for 15 experts (Table 1). The draft thus shaped was termed Draft-I.
Second Stage: Analysis and discussions on Draft-I draws attention to the speculation that respondents might have ranked the items as 'Useful, but not necessary' instead of 'Essential' without understanding that items ranked as 'Useful, but not necessary' will be removed from final scale (Lawshe, 1975). This misperception between 'Useful, but not essential' and 'Essential', also resulted in low CVR. Therefore in second stage the CVR is re-calculated combining both 'Essential' and 'Useful, but not necessary' ratings to give a combine value of CVR E+U (Kawachi, 2014). The CVR E+U of scale is calculated to be 0.62, which is more than critical value of 0.49 at p<0.05 level for 15 experts at 0.05 level. The draft shaped after second stage was termed Draft-II.
Third Stage: Only 4 experts responded at this stage. This low response may have been avoided by providing background of this research study and explaining the three constructs to the experts. However, the validity process expects un-influenced opinion on the items. CVR E+U of the revised scale was 0.68. Additionally, the calculated value of CVR E+U is 1.00 for most of the new items ( Table 1). The draft shaped after this stage was termed Draft-III. Fourth Stage: At this stage, 8 items (item no. 2, 3, 13, 20, 27, 28, 29 and 34, from the third stage) were omitted owing to their low CVR E+U value. The average calculated value of CVR E+U for 26 items was 0.88, which is more than the critical value of 0.42 at p<0.05 level for 20 experts. This was considered to be satisfactory for further statistical tests.

Exploratory Factor Analysis and Reliability of the Scale
The 26-item scale was subjected to reliability test using two methods that showed Cronbach's alpha at 0.897 and Guttman Split-Half Coefficient at 0.790, which provided confidence that the items in the scale are interrelated and are measuring the same attribute, i.e. Attitude towards OER. With this we were interested in analyzing the three constructs of the scale: Awareness, Sharing and Adaptation.
Before undertaking factor analysis, we conducted Kaiser-Meyer-Olkin Measure (KMO) of Sampling Adequacy. Kaiser (1974) recommended that KMO values between 0.8 and 0.9 are great, and Table  2 shows KMO value of 0.82 for the data used in the study. This gives confidence that the sample size is adequate for factor analysis. Also, the Bartlett's test of Sphericity reveals that it is highly significant (p<.001), indicating that there are some relationships between the variables. Factor analysis of the 26-item scale using principal component analysis method assuming three factors confirmed the assumption showing only 21 items with factor loading more than 0.5 or greater.

Scale Optimization
The correlation between these three factors (Table 4) revealed that factor 1 and 2 is positively correlated with moderate coefficient value 0.46, which is also significant at 0.01 level. On the other hand, factor 3 has a very low correlation with both factor 1 and 2. Cronbach's alpha for factor 1 is 0.89, which is good enough. Furthermore reliability coefficient of factor 2 and 3 are measured as 0.71 and 0.61 respectively. Thus, the correlation between all these factors and the reliability coefficient revealed that factor 3 is not correlated with factor 1 and 2; however, the Cronbach's alpha with 0.61 is acceptable but not good enough. Because of this, we decided to discard the factor 3 (with 3 items) from the scale. Once it was decided to use the two factors with 18 items, we conducted inter-item correlation for both factors (sub-scales). The standardized Cronbach's alpha for the 13 items in the Sharing scale was 0.898, while for the five items Adaptation scale was 0.734. In Tables 5 and 6, the values in the column labelled Corrected Item-Total Correlation are the correlations between each item and the total score from the questionnaire. In a reliable scale all items should correlate with the total. We used the advice of Field (2009) to look for items with less than 0.3 to identify, if any item does not correlate very well with the overall scale. Interestingly for all the items, item-total correlations are above 0.3.  Once we were confident about the sub-scales, we conducted inter-item correlation for all the 18 items in the scale, and only one item showed correlation value of less than 0.3 (i.e. item 1 from the sub-scale Adaptation, with 0.170). Further, the result indicated that deleting the item from the scale would increase the reliability score to 0.897. Therefore, the final number of items in the scale is 17 (with 13 items for Sharing and 4 items for Adaptation). For the sub-scales, the reliability co-efficient (Cronbach's α) is 0.893 and 0.715 for Sharing and Adaptation, respectively.

Confirmatory Factor Analysis
While the optimization process and reliability tests revealed a 17-item scale with high validity and reliability, we also conducted Confirmatory Factory Analysis (CFA) on the data set, as in the beginning we had assumed three factors based on literature review and conducted the exploratory factor analysis. The process of scale development resulted in a two-factor scale, and we wanted to analyse if the two-factor model is a good fit. Thus, we followed Structural Equation Modelling (SEM) using SPSS AMOS. The SEM is used commonly to test whether measures of a construct are consistent with the researchers' assumption of the nature of the construct. Figure 1 shows the Path diagram of the CFA, which shows that the two constructs (sharing and adaptation) are correlated. There are several ways of determining model fit, and the common measure is to follow the chi-square (c 2 ) goodness of fit. In this case the c 2 value of 204.548 at 118 degree of freedom is high rejecting the model fit. However c 2 is affected by the sample size, therefore c 2 /df ratio, which in this case is 1.73, is used as a measure of good fit. Kline (2005) recommended that as a rule of thumb, c 2 /df values of 3.0 or less signify a good fit of the model. We further used the Root Mean Square Error Approximation (RMSEA) statistics for analyzing the model fit. For RMSEA, choosing a proper cutoff value is critically important, and a widely used convention is that ≤0.05 refers to close fit, ≤0.08 mediocre fit, and > 0.10 poor fit (see, e.g., Browne & Cudeck, 1992;MacCallum, Browne & Sugawara, 1996). For our data the RMSEA value is 0.08, which indicates the model presents a mediocre fit and the proposed two-factor model is plausible.

Discussion and Conclusion
The development of ATOER scale with 17 items and two factors (Appendix 1) following a consultative process with valid and reliable statistics shows that the scale can be used to measure what it is supposed to measure, i.e. attitude towards OER. While we started the analysis with only 117 responses, and considered it may be a limitation, the final scale with 17 items satisfactorily falls within the acceptable limit of sample size (Tinsley & Tinsley, 1987). The CVR score for the final scale is 0.9, and the Cronbach's reliability co-efficient α is 0.897. It is interesting to note that, based on the literature review, we assumed a three-factor model of attitude towards OER, and during the reliability tests, we dropped the items related to awareness. In retrospect, this looks obvious as mere awareness may not have influence on the overall attitude, and those sharing and adaptation behaviour are certainly manifestation of attitude towards OER. This has significance for the OER practitioners, as only advocacy and increasing awareness of OER may not help promote the cause of OER.
The two-factors model using the data in the study revealed a mediocre fit, and therefore, we proffer that the scale is acceptable on the basis of its other psychometric properties. However, further tests may be needed with more data and other contexts to test the model fit. We could have further conducted modification indices to come-up with an acceptable model fit. However, as the exploratory factor analysis and reliability tests are sufficient for any scale development, we assume that the mediocre fit presents a plausible model that could be further tested by other researchers using the scale developed in this study.
In order to use the scale in practice, several demographic variables may be used to understand difference due to gender, experience, age, discipline, etc. However, the important use of the scale would be to know how a group of teachers in an educational institution are disposed to OER, and therefore, whether they would take steps to use and contribute to OER. The development of the scale is within the context of a wider research being currently conducted to understand why some teachers share their work and others do not. As such, one of the ways the scale would be used is to explore the possibility of predicting who would be more willing to share educational materials with open license. When used with other psychological aspects, such as motivation of individuals, the scale might help to demonstrate better results related to who would share learning materials. However, institutional policies may also have some impact on sharing behaviour, while individuals may have positive attitudes.
Generally there is an inherent assumption in attitude measurement that we may reasonably expect respondents to accurately reflect their own held beliefs (Thurstone, 1938). In attitude studies, 'social desirability' aspect of the respondents to show a positive disposition may also influence the outcome of the scale, and could be investigated by adding a 'social desirability scale' such as that developed by Strahan and Gerbasi (1972) alongside the present 17-item ATOER. While the methodology of the scale development shows the rigor, investigation on a larger or different sample may be warranted to further validate the scale.
The scale will help institutions to plan use of OER in teaching and learning by identifying positive and negative faculty attitudes. Policy makers and planners will be in a better position to manage change and implement an organization-wide OER strategy with an empirical understanding of the ground realities. As attitudes naturally change over time, it is possible for institutional administrators to change any negative pre-disposition among faculty through interventional information communication, training and implementing projects related to OER.

Appendix 1: ATOER Scale
This Attitude towards Open Educational Resources (ATOER) scale is intended to assess attitude towards OER. Please indicate your level of agreement or disagreement by putting a ü mark in the appropriate column. SA = Strongly Agree, A =Agree, UD = Un-decided, D = Disagree and SD = Strongly Disagree.