Harnessing the Use of OERs in Teacher Education

Open educational resources (OERs) are used by teachers to help find resources to integrate into their classroom and find resources for teaching and learning (UNESCO, 2019). The purpose of this study was to investigate the OERs that pre-service teachers (PSTs) used within their field experiences during one semester in a teaching methods course and determine whether PST training improves the inclusion and citation of digital OER resources. To address the purpose of this study, the researchers collected survey data and lesson plans before first intervention, after first intervention, and after a second intervention. Findings suggest PSTs may benefit from learning about OER resources, how to use them when planning lessons, and how to cite them properly in lessons within all subject areas.


Introduction
According to the UNESCO (2019), Open Educational Resources [OER] "are teaching, learning and research materials in any medium -digital or otherwise -that reside in the public domain or have been released under an open license that permits no-cost access, use, adaptation and redistribution by others with no or limited restrictions." OER resources can be used by pre-service teachers to find materials to incorporate within lesson planning (i.e. streaming videos, assessments, labs and activities). The purpose of this study was to identify whether pre-service teacher (PST) training on OERs improves the inclusion and citation of digital resources within their written lesson plans. The goals of conducting two trainings within the same class were to help PSTs improve the methodology in how they find resources and improve the quality of the resources used within their lessons. The following research questions shaped this research: The following research questions shaped this research: • Do trainings on OERs for PSTs improve the incorporation of digital resources within lesson plans or the quality and citation of resources used?
• Do trainings on OERs change PSTs methodology on how they are finding resources based on their reference point?

Relevant literature
Williams' (2010) effective teaching framework consists of the following six practices: professionalism, professional knowledge, learning environment, assessment, instructional planning, and instructional delivery. According to Clark and Peterson (1986), teacher planning is critical within pedagogical practices as it helps teachers visualize the future, construct frameworks to guide their actions, and describe what they will do and say. Lesson planning is a process where teachers engage in metacognition, or thinking within their own thoughts, and devise a plan to teach a specific group of students, at a specific time and within the context of a specific environment (Savage, 2015). Lesson planning requires teachers to deliberately make decisions that directly impact student learning and engagement. Teachers should begin by asking themselves what makes a good lesson plan. Research identifies that good lesson plans are detailed in nature, incorporate accommodations, and include a variety of activities which results in effective teaching and higher student achievement (Hoover & Hollingsworth, 1975;Wilen & Kindsvatter, 2000;Brophy & Good, 1986;Wood & Miederhoff, 1988).
Since planning lessons is a complex and challenging process, teachers should be trained within their undergraduate courses on the tools necessary to be successful in order to link curriculum with student learning (Marzano, 2017;Clark & Yinger, 1987). This can be a challenging task due to the multiple models of lesson planning that exist within undergraduate education professional programs, which include course curriculum and unit planning, many often using templates with the focus on the learning outcomes (John, 2006). Throughout the review of the literature, a topic of study with limited resources remained in the area surrounding research pertaining to the incorporation of OERs by PSTs (Clark & Yinger, 1987;John, 2006;Marzano, 2017;Savage, 2015).

Metacognition and lesson planning
When teachers are planning lessons, they are engaging in metacognition which is defined by Zimmerman as the "awareness of and knowledge about one's own thinking" can be improved through practice, guidance, and encouragement (2002, pp. 65). Students can practice this skill by writing and critiquing lesson plans and revising them based on feedback from teacher educators.
When lesson planning, PSTs need to select appropriate strategies, monitor the quality of their plans, diagnose any problems, and revise lessons based on their review.
Within this study, the authors first taught PSTs about the use of and selection of OERs to assist in the selection of appropriate resources for their lessons. The authors then engaged PSTs in critiquing and revising a poorly cited and low-depth lesson plan to identify common misconceptions. Afterwards, students evaluated a well-written lesson, which modeled accurate citations and details in order for them to become aware of how the task is performed.

Planning with OERs
OER materials can provide no-cost access to information in any medium, which can include videos, podcasts, lesson ideas, textbooks, or even full courses (Sparks, 2017). While OER is commonly discovered online, it is different from online learning (UNESCO, 2011). The defining feature of OER content is the license of use permitting others to use freely and adapt the information (UNESCO, 2019). Oftentimes creators utilize creative commons licensing for sharing, which is picked up by common search engines such as Flickr, Google, and Wikimedia. These licenses allow creators to retain the rights to their materials, but also grants users specified allowance to the works in a "human readable" version. With this in mind, OERs create an opportunity for educators to seize new materials and ideas to incorporate in lessons to the benefit of the students without worry of infringing on the creator's rights.

Benefits and importance
Many teachers will user OER content in an ad hoc manner supplementing existing curriculum (Spark, 2017). However, to do so, teachers must first be able to discover OER content. There are a number of websites that attempt to index the number of resources available on the web. Two of such websites include OER Commons and MERLOT. These attempts to make content discovery easier by aggregating and deploying metadata, which is a needed piece to promote OER inclusion in teaching (Lane & McAndrew, 2010). Those who have included OER have seen positive influences of the use. Kimmons (2015) found that OER textbooks maintained a high-quality rating. Choi and Carpenter (2017) saw in the higher education realm a highappreciation from students for the use of the material with minimal complaints (eye strain/fatigue from digital only content) with little effect to the student learning outcomes. These examples point to reasons PSTs and in-service teachers may consider using OERs within the K-12 environment.
The quality of an OER depends on the resources PSTs choose to use, and how they are adapted within their lesson plans (UNESCO, 2019). As a PST, planning lessons can become challenging without having a course text to begin the process. The OERs can serve as a low-cost starting point for PSTs to find resources when creating lessons prior to their student teaching or more in-depth teaching experiences. Since PSTs do not have the access to curricula materials as in-service teachers do, OERs can serve as a way to find textbooks, lessons, and supplementals for their planning. Allowing teachers the autonomy to make decisions about curriculum materials and their use in the classroom is crucial as it is a skill they need to do effectively (Lloyd, 1999). Therefore, providing this support for PSTs is especially valuable.

Barriers
Although OERs can provide avenues for PSTs to engage with content that is more specific, there are barriers that exist. Typically, teachers will rate OER materials highly once they are aware of their existence (Kimmons, 2015;Nerantzi, 2013). Nerantzi (2013) found 75% of teachers would consider or plan to introduce OERs in classes once they learn more about the availability of them. However, gaining teacher awareness to these accessible resources is the first hurdle, as many are still only aware of options such as YouTube or TED Talks (Farrow et al., 2015;Misra, 2013;Nerantzi, 2013). There is reason to believe that OER use is becoming more widespread in higher education due to the rising costs of education (Lieberman, 2019). As PSTs understand the use and benefit of OER materials in their higher education experience, there is reason to believe they could translate this use in their lesson planning.
Other barriers identified with the incorporation of OERs in courses include instructor uncertainty with language barriers or a lack of translation of the material for the culture; however, these findings tend to be most prominent in areas outside of the United States (Pirkkalanien, Jokinen, & Pawlowski, 2014). Teachers also tend to question their ability in assessing OER content, which is where peer-reviewed material and supporting metadata can assist in discovery (Lane & McAndrew, 2010;Pirkkalanien, Jokinen, & Pawlowski, 2014). Teacher confidence in using OERs also seems to be divided among those with digital competence and experience teaching online (Ramirez-Montoya, Mena, & Rodriguez-Arroyo, 2017). Many of the barriers to OERs for PSTs are challenges that can be overcome through proper training and practice.
With this in mind, a goal of this study is to inform the design of PST education to promote teachers' productive use of instructional materials with the incorporation of OERs. The study also investigates how two different supports for critiquing lesson plans can help pre-service middle and high school teachers engage in the authentic task of planning.

Methodology
In order to answer the research questions, the authors surveyed PSTs regarding their lesson planning within one education methods class three times after approval of the institution's IRB and permission from the course instructor as well as voluntary consent from the course participants. There were no identified risks involved in this study and all participants were informed that they would be able to withdraw of their involvement at any time.
The authors conducted two forty-minute interactive teaching seminars on how to utilize OER websites to aid in the search of lesson plan resources. Following both OER trainings, pre and post student lesson plans and pre and post survey data were analyzed. The analysis focused on the number and types of OERs used within the student lesson plans, how OERs were incorporated, and student perspectives regarding how they searched and analyzed online resources. All data collected during the study was anonymized and kept private from the course instructor to ensure that student grades would not be impacted.

Participants and setting
Undergraduate students chosen for this study were enrolled in a methods class within their second year education program. Eight prospective teachers were enrolled in a semester long methods course at a midwestern private university in the United States and agreed to volunteer to participate in this study. This class consisted of four secondary (two female, two male) and four middle childhood (three female, one male) pre-service teachers. This particular private university's enrollment totals more than 2,243 students. The PSTs all had varying backgrounds and reasons for wanting to teach in their particular content areas of study.

Context about the course
As part of their methods courses, PSTs in this course experience their third in class experience in the classroom. This course is taken during the students' fourth semester in the education program. The PSTs are given a common lesson plan template that they use from their first education class until the end of the program. This template is divided into seven sections that include: lesson focus, assessment, materials, instructional procedures/learning tasks, differentiated instructional supports, accommodations, extension/homework, and a final reflection (Appendix A). For this particular class, students were asked to write three lesson plans. The instructional procedures/learning tasks represent the largest section of the lesson plan where PSTs describe in detail the instructional strategies, learning tasks, student activities, teacher activities (co-taught or general classroom), and the time for each activity. This section of the lesson was of focus when conducting OER training and evaluating PST lessons.

Training procedures
The authors, or researchers in this study, which consisted of the university librarian and an associate education professor, led two forty-minute trainings during the practical education class on OERs after they had written their first lesson plan for the course. The first training procedures were selected with metacognition for learning and overcoming identified barriers toward the use of OERs. Because OERs can exist anywhere, the first training focused on two open educational website aggregate resources: OER Commons (https://www.oercommons.org) and Merlot (https://www.merlot.org/). Once the first training was complete, the instructor of the course added these two resources to the classroom's online course learning management system page for easy student access later. This training session went as follows: 1) The training began with a pre-assessment to discover what types of websites students already use to find their lesson plan resources. This survey was done through an open response question and had students write on sticky notes all of the resources they used.
2) The university librarian then used the computer and projector to demonstrate the web resource OER Commons. A step-by-step procedure was demonstrated to students on how to use this resource to search for lesson planning materials related to their content area of focus. 3) Time was then provided in order to allow students to navigate OER Commons to find a lesson resource related to their next lesson plan topic as the instructors walked around the room and helped individual students search using OER Commons, and 4) Steps 2-3 were then repeated with the free web resource Merlot. Students were then tasked with utilizing the resources learned in order to write their second lesson plan. The end of the lesson plan also included a survey to have students reflect upon their searching strategies and resources used in order to determine growth (Appendix B).
A week after the first training was completed, students submitted their second lesson plans to their course instructor, which were then shared with the researchers. During the following week, the researchers reflected on the OER needs of the class by reviewing each individual student's lesson plans and surveys to see if any improvements were made from their first attempt. After reviewing the surveys and the lessons for the second lesson (after the first OER training), the researchers noticed that the survey responses did not align with the lesson plans. Many students stated they used particular websites to search for and find content to use within the lesson plan surveys but did not describe materials or cite these resources within their lesson plan, demonstrating a need for a second intervention focusing on citing sources in lesson plans.
The researchers then led a second forty-minute training during this same class period focusing on citing references within their lesson plans. The second training was selected for a continuation and reinforcement of the metacognition from session 1. This training also targeted specific weaknesses of the class participants in order to improve their use of OERs. The first twenty minutes of the class required PSTs to analyze and critique a poorly written sample lesson plan to help them identify what important information is required when writing their lessons. This strategy reinforces metacognitive awareness as PSTs were asked to review the lesson as if they were a substitute teacher and see if they would feel confident and ready to teach this lesson to students that day. All of the students did not feel comfortable teaching this lesson (Appendix D) due to its limited amount of information and resources provided. The second half of the training used a sample lesson plan resource found within OER Commons to show students how they can complete their lesson plan template while citing sources within their materials and lesson descriptions. The purpose of this training was to illustrate how to properly cite the resources to give credit to the author, allow for easy access at a later date, increase lesson description, and aid in future teaching.

Pre-assessment of resource types
When starting the first training, researchers asked PSTs where they go to find their lesson plan resources. Students were able to list more than one resource. Data collected from their responses is displayed within Table 1. After students identified the multiple web resources they used to find lesson plans, the researchers reviewed each website to determine whether it met the criteria to be classified as an OER. Each of the eight students listed more than one source. This resulted in seventeen responses with ten different resources identified. Of the ten resources identified, only four were classified as OERs (40%), which consisted of seven out of the seventeen responses. Three out of eight students stated that they utilize Google and type in their lesson plan topic as a keyword, while two students other expressed they were willing to pay for resources. Both instances relate to the ease and convenience of the search.

Data analysis procedures
The lesson plan analysis used in this study required PSTs to summarize the websites that they used within their lesson plans, and incorporate these resources within their lesson planning template (Appendix A) while analyzing their process to complete the template through a survey (Appendix B). The researchers analyzed the qualitative data through the student lesson plans and survey by searching for patterns among student lesson plans (McMillan & Schumacher, 2001). Four phases were employed during this qualitative analysis: 1) researchers looked for patterns among individual student lesson plans, 2) data gathered was organized and categorized, 3) patterns were refined by determining the trustworthiness of the data, and 4) themes were synthesized. The researchers assigned numerical frequencies found from individual lesson plans in order to compare student lesson plans. In order to ensure triangulation and increase trustworthiness among the data, the researches ensured to incorporate multiple forms of data collection: two student surveys and student three student lesson plans (Lincoln and Guba 1985).
PST lesson plans and surveys were compared among the pre-and post OER training (Table 1). Within the lesson plan survey students responded to the following questions: Where did you find your lesson plan resources?, How did you locate your lesson plan resources?, and Rate the ease of locating your resources for your lesson plan on a Likert scale from 1 (very frustrated) to 5 (very accessible).
After the second training on citation with OERs, PST lesson plans (lesson plan three) were compared with their previous lesson plans (lessons one and two) to compare resources used and citation of these resources. The university lesson plan rubric was used (Appendix C) when analyzing PSTs three lessons. This lesson plan rubric was adapted for this study to only focus on the materials (section III) and the instructional procedures (section IV) in order to identify whether student scores showed any improvements in the details of their materials, lessons, citation of resources used, and their inclusion of OERs.

Results, analysis, and discussion
Survey results for the Likert scale question were collated and analyzed by the authors manually by using Excel in order to identify a correlation between the structure of lesson plan written by students and the trainings provided by the researchers to answer the two research questions within this study.
To address the first question: Do trainings on OERs for PSTs improve the incorporation of digital resources within lesson plans or the quality and citation of resources used?, a correlation was used to compare PST lesson plans across the three sets of lessons submitted as they related to the two forty-minute training session treatments to identify if the treatments had any impact. The survey (Likert-scale) and lesson plan rubric scores were calculated to find the Pearson's r coefficient. This correlation coefficient was used to determine if there was a significant relationship between the two training sessions and student lesson plan details, resources, or citations (Chen & Popovich, 2002).
Lesson plan surveys were analyzed to see where students went to find their lesson resources. The responses by students were changed into a percentage to represent the amount of OER use in their search. For example, if a student listed Pinterest as their only resource they would have scored a 0% in OER usage, while a student who listed Pinterest and OER Commons would have scored a 50% as one out of the two resources was classified as an OER. This data was collected for each individual student and an overall change in their percentages from lesson plan 1 to lesson plan 2 is displayed within Table 2. Prior to the OER training, no students used OERs within their first lesson plan. Researchers found that a majority of students did not use any online resources. They simply made up the lesson based on their previous experiences in class and the standards they chose to teach. Student lessons were very vague and limited in materials. For example, a student's lesson included the following items within their lesson procedures: 1. Entry question, 2. Take notes, 3. Have students write down homework, and 4. Clean up classroom. The student did not include any description or other resources that would explain the entry question students are to answer, the notes they are to write down, or what their homework consists of.
After the first intervention on using OERs, two out of eight students (25%) incorporated the use of OER resources within their lesson plans. The intervention also resulted in three students stating an increase in their ease of finding resources within the Likert scale survey, although one did decrease their rating but did not use OERs in their lesson. This decrease in rating could be due to a change in the content standard of interest. Comparing the change in pre to post ease of use, with a p value of <0.05, the correlation (r) = 0.5345 demonstrated a moderately positive correlation between the first training on using two OER websites to find lesson plans, making it easier for students to find resources. Since r 2 = 0.286, the correlation is not significant, most likely due to the number of the participants.
After the second training, which focused on using OERs to write lesson plans and cite resources within their lessons, Table 2 also displays a second analysis completing the same processes as identified above.
Comparing the use of OERs from the second to third lesson plan yields a correlation (r) value of 0.4174, (r 2 = 0.174) at a P <0.05, representing a positive correlation that is also not significant. This correlation shows that there was a weak relationship between giving an intervention to students on how to cite OER resources within their lesson plans and the types of resources students used in their subsequent lesson. Second, comparing the Likert scale values of the second training and PSTs ease of searching for lessons yielded an r value of 0.1826 (r 2 = 0.033) at P<0.05, showing a weak positive nonsignificant correlation. The strength of this correlation score may be impacted by student number seven who expressed it was more challenging to search for resources. This student did not incorporate any OERs and only used Google as a resource.
Next, the lesson plan scores for lessons one and three were compared. The change in scores within Table 2 were found through individual analysis of each lesson using the lesson plan rubric (Appendix C) by each author individually, and then authors compared scores to agree on one final score for each PST. Lesson plans were scored out of ten points in three areas: materials, OER inclusion, and lesson plan procedures. These scores were compared within lessons one through three to identify if the trainings had any impact on student's lesson planning with the incorporation of OERs and proper citation of OERs within their lessons.
Comparing the scores of lesson plan one to lesson plan two yielded a correlation coefficient value (r) at 0.7977 (r 2 = .636) at P<0.05, representing a strong positive correlation that is significant. This means that the first training on OERs increased the use of OERs within their lessons, citation within their materials, and increased the details within their lesson plans while citing sources.
Next, comparing lesson plan two with lesson plan three represents a correlation of r= 0.315 (r 2 = .099), a weak positive nonsignificant correlation. At this point, many students already knew about the OER resources from the first training, therefore the largest change identified within these lessons included an increase in citations of their resources and more elaboration on the instructions within their plans. Often times, low evaluations scores sourced from student lesson plans claiming to use multiple sources, but the plan lacked detail, was vague, or did not specify how resources were used.
While this study focused on instructing students on the use of two particular OER resources (OER Commons and Merlot), the researchers primary focus was on enhancing awareness of OERs rather than mandating their use. It is important to note that other non-OER or free resources were not discouraged in this study. The researcher's main focus was on the improvement of the details within student lesson plans, utilizing resources within lesson plans and knowing how to properly cite those resources within their plans in order to easily find these resources with future access in mindquant.

Concluding remarks
Training PSTs did improve their use of the open educational resources in lesson planning. Prior to intervention, no student in the class knew of or used OERs in their lesson planning process, which is consistent with other studies that demonstrate knowledge of OERs is the biggest barrier to using OERs (Nerantzi, 2013).However, not all students utilized the new search methods introduced to them and continued to rely on simple keyword strategies in Google to find supporting lesson plan materials if any was used at all. This may be due to a perception that navigating and searching multiple OER interfaces, such as MERLOT and OER Commons, is more time exhaustive and challenging than the common Google search. Those who did include OERs tended to have the most improved lesson plan rubric scoring by the end of lesson plan three.
While the findings of this study are encouraging for those who wish to integrate more OERs in PST's lesson plans, the study's limitations should be noted. Because of the size of the education program at the selected institution, the sample size was very small consisting of just the eight students enrolled in the course. Therefore, this study is not generalizable and should be conducted again with a larger sample size. In addition, the class's makeup consisted of a variety of PSTs grade level focus (middle and high school), and content specialties such as history, English, math, and others. It is possible that certain subject areas experienced an easier time locating materials in the selected interfaces used in instruction skewing some students more likely to use the resources than others. The study could be replicated to include the grade level areas and subject matter of the lesson plan as variables of consideration.
Based on these results, future researchers may be interested in exploring student analysis of resource quality and their relevance to lesson plans. This study sought to understand the types of resources used in the lesson plan but did not emphasize or analyze the quality or relevance of the resources the student selected. Future researchers may also want to see if the level of detail in a lesson plan or the quality of resources cited in lesson plan have any relationship to the learning of PST's students in the classroom. This study did rate the level of detail a student used in their lesson, but because these students were not teaching others yet, there would have been no way to measure the learning of the PST's students.