Instructional Design Considerations for Flipped Classroom

As technological advancements continue to revolutionise the realm of education, use of technology in both K-12 and university classrooms paves the way for making flipped classroom a trending pedagogical model. Flipping a course simply means reversing a traditional model of in-class lecture followed by practice and homework. Research has shown that use of flipped classroom holds promise for students’ success and satisfaction as long as curricular activities of a flipped course are designed and sequenced through some steps depending on tenets and nature of flipped classroom as well as principles of instructional design models. This study aims to shed light on some considerations for designing content, pedagogy, materials, delivery, and assessment prior to flipping a course. Each step in the process of flipping a course is therefore explicated through a systematic review of 78 studies that focus on implementations of flipped classroom in K-12 and university classrooms.


INTRODUCTION
Technological advancements affect our lifestyles and educational practices. As the 21st century leads to globalisation of ICT by the production of important amount of new knowledge, education systems need to shift from teacher-centred ones to more student-centred ones. The reflection of technological advancements on educational methods may account for the use of audio-visual tools for educational purposes. In conjunction with the developments in technology and the change of teaching and learning paradigm, a new pedagogical approach called flipped classroom has appeared (Lo et al., 2017). One of the fundamental characteristics of the flipped classroom is to significantly reverse pre-and in-class learning activities by allowing the teacher to devote more time and energy to deepen students' understanding in in-class time.
Gaining in popularity in recent years (Enfield, 2013) as a technology-enhanced pedagogy, flipped classroom relies on some components like videos, reading texts, and other resources out of the classroom (Hawks, 2014) where students are expected to establish conceptual knowledge through the given pre-class materials to get prepared for in-class time in which they are supposed to make sense of the material by means of their active engagement in activities in lieu of being passive listeners of traditional direct lectures (Butt, 2014). In traditional direct lecture, knowledge is conveyed to the student by the teacher and in order for students to internalise and apply it, homework is assigned (Bishop & Verleger, 2013). Traditional direct lecturing has faced a number of challenges in meeting the needs of students and enhancing their knowledge and skills that may enable them to be global competitors of the 21 st century. In order to break this traditional direct lecture-based teaching and transform it into a self-directed and active learning approach, flipped classroom could give educators a chance of replacing traditional teaching methods with a more student-centred approach in which students collaborate with peers about some activities such as tasks or projects, engage deeply with content, enhance their thinking skills, and receive feedback about their improvement (Hamdan et al., 2013;Kong, 2014;Missildine et al. 2013). Being quite distinct from traditional direct lectures, flipped classroom provides students with valuable classroom time for activities in the form of discussions, projects, or team work that demand higher order thinking skills (Pulley, 2014) on the basis of gained conceptual knowledge out of the classroom.
Flipping a course does not boil down to the use of videos and homework; on the contrary, it mainly centres on putting attention on students (Bergmann & Sams, 2012). In other words, flipped classroom aims at creating a student-centred environment where the emphasis of the provision of meaningful activities that enable students to gain higher order thinking skills is highly laid. Flipped classroom, a blended model of learning, might be different in various classrooms depending on the teacher, content, the use of digital resources, tools, and technology (Osguthorpe & Graham, 2003). Since coupled with the current educational technology providing time and rich resources for the teacher, flipped classroom offers the teacher the chance of integrating digital resources such as PowerPoint with audio, lecture videos, online reading texts or in PDF format into the content of a flipped course (Pulley, 2014). As well, flipped classroom gives the teacher the opportunity to create pre-and in-class activities and assignments for students based on their intellectual, physical, and cultural diversities (Larcara, 2015). Based on active and individualised instruction, flipped classroom also enables the teacher to recognise how much students perform well in the content and apply it on deeper level while they share their ideas with each other and offer different perspectives on the content through collaboration and cooperation with their peers (Sams & Bergmann, 2013).
Peer interaction and collaboration are the most beneficial aspects of flipped classroom when compared with traditional direct lectures where they can have little interaction with each other in class time and may have a connection out of the classroom if they could make an effort to do so (Larcara, 2015). Last but not least, flipped classroom claims to help students improve their innovation and task orientation (Strayer, 2012), information literacy, critical thinking skills (Missildine et al., 2013), and performance on assessment (Mason et al. 2013). The use of quizzes in flipped classroom shows how much students are successful in integrating conceptual knowledge into in-class activities, which is often times very hard to measure if students can integrate conceptual knowledge in a traditional classroom through an exam (Swart, 2017). With flipped classroom, the teacher will also have the chance of walking around the classroom and observing collaboration and interaction among students in order to better assess students' understanding of the content. In addition, there are a number of formative measures the teacher could use such as think-pair-share questions, exit tickets, or oneminute papers (Black et al., 2017;Gardner, 2017) to check students' understanding of pre-and in-class activities.

Instructional Design
The effectiveness of teaching depends on effective planning of instruction and design of instructional activities. The process of instructional design provides a framework for the teacher to plan, develop, and adapt instructional activities according to the needs of students and the requirements of course content (Jones & Davis, 2008). In the field of instructional design, psychologists and educators have started to look into effective ways of planning and implementing instruction. There are a number of researchers who have studies into learning theories, instructional theories, and systematic approaches to instruction (Carr-Chellman, 2015;Gagne et al., 1992;Gros et al., 1997;Morrison et al., 2007;Reigeluth,1999).
The aim of instructional design is to help the teacher understand what is significant in creating learning environments and offer some guidelines about how to optimise students' learning process. Instructional design process, which is prescriptive by its nature, is generally based on models. Instructional design models offer some advice concerning the design of learning activities and support of students (Elen, 2016). It is possible to use instructional design models with varying degrees of implementation in many settings. Besides, since an instructional design model is effective in planning instruction from the perspective of students, through step-by-step process it helps the teacher to specify students' needs, define what is to be learned, determine a sequence of course content, outline actual design, develop or adapt learning resources, and assess the overall instruction process (Hains, 2000;Morrison et al., 2001). Instructional design models are open to modification based on students' changing needs and advancement of educational technology. Thus, the integration technology into instruction process means presenting different learning opportunities that appeal to students (Jones & Davis, 2008). In addition to this, when there is the integration of instructional technology with a wellplanned curriculum that covers proper teaching strategies, students' learning could be improved.
Similarly, when designing activities for a flipped course, the teacher generally should take thoughtful steps to ensure that assessment, teaching techniques, and curricular activities all align in terms of practical and sound educational theories. According to Brewer et al. (2001), planning and preparation of a course directly impact the effectiveness of that course and really impedes learning of students. In making design decisions about how to flip a course, course goals must be used as a strategic planning step to guide the teacher in selecting pre-class, in-class materials, and activities together with a suitable evaluation plan aligned to other course components (Swart, 2017;Zappe & Litzinger, 2017). In a qualitative study investigating faculty perceptions of flipped classroom, it is highlighted that alignment of pre-class activities with in-class activities is regarded as a critical factor in the success of flipped classroom (Zappe et al., 2015). In addition to this, what matters most is the design of clear and understandable course content for students' success. Design process requires a firm connection between what students are supposed to do in-class time and what they need to do pre-class time (See & Conry, 2014).
Because of its flexibility (Bergmann & Sams, 2012) and intricacy in the process of both design and implementation , flipped classroom could be hard and daunting for the teacher to plan and develop flipped classroom-based curricular activities, thus the availability of a design model is crucial to lead the teacher to implementation of flipped classroom at ease. With regard to designing models for flipped classroom, a group of researchers initiate the development of a design model from the synthesis of blended learning and flipped classroom literature that are mapped onto ADDIE process (Analysis, Design, Development, Implementation, and Evaluation)  in order to make contributions to solid employment of flipped classroom in higher education. Initial design of the model is composed of the analysis of goals, content, students and technological environment, the features of content such as sequence, hierarchy, and interactivity, and assessment of pre-class activities with formative assessment instruments. The final design model for flipped classroom at the course level covers the following steps as analysis, online design, online development, face-to-face design and development, and implementation. It is, however, highlighted that since the design model is developed from an algebra course, confirmation in various courses is needed. In another empirical study investigating key principles of how flipped classroom could be put into practice, it has been clearly stated that there are nine key principles, the first three of which are adopted from Brame's (2016) work and the remainders of which are constructed as a result of the study carried out by Kim et al. (2014). These principles are generally comprised of providing opportunities for students' pre-class time, an incentive for them to be ready for in-class time, a mechanism for their assessment of understanding, connections between pre-and in-class activities, well-established guidance, enough time for tasks, promotion of a learning community, feedback on individual or group work, and familiar and easy technologies for access.
The literature review reveals that there is a scarcity of research both detailing design principles and models that offer a viable solution guiding the teacher into designing and implementing flipped classroom (Cresap, 2017;Kim et al., 2014;, hence with a view to shed light on some considerations for designing content, pedagogy, materials, delivery, and assessment, this study seeks to answer the following research questions:

METHOD Article selection process
Web of Science (WOS), ERIC, ScienceDirect, SCOPUS, IGI Global, Wiley Online Library, Emerald Insight, and Sage are considered to be major databases (Taylor et al., 2003), thus they were used to retrieve relevant studies for review process. The search option was used in the aforementioned databases with the following query string: "flipped classroom" OR "flipped learning" OR "inverted classroom". The search came up with 350 studies in total as the end of 2019 (since 1980). Upon applying the parameters such as the document type as "articles" and "language" English and choosing educational categories (education & educational research, education special, education scientific discipline, psychology educational) in the search filter, there were 125 articles that fit those parameters. The last search was performed on the 1st of January 2020. Full texts of these 125 articles were downloaded and their suitability for the current study was examined by the researcher in collaboration with an external researcher having expertise in conducting systematic reviews in social sciences. In line with the criteria of inclusion and exclusion in Table 1, the suitability of the 125 articles was evaluated. As a result of the evaluation process of collected articles by the researchers, 78 articles out of 125 were found suitable for the purposes of this study.

Data coding and analysis
The articles that met the criteria of inclusion were coded by the researcher with the external researcher. In encoding process of the data, Microsoft Excel program was utilised. Initially, randomly selected twenty articles were separately coded to compute coding reliability. Results of coding process revealed that inter-rater reliability was 0.92 with Cohen's kappa analysis. Upon ensuring coding reliability, the remaining articles were independently coded by the researchers. Following the completion of the coding process, the researchers reached an agreement by discussing the codes on which they had some disagreements.
Content analysis was conducted to analyse the data. Content analysis is a process that covers the organisation, classification, comparison of texts and development of theoretical results (Cohen et al., 2005). In the process of data analysis inductive approach, which is suggested by Miles and Huberman (1994), was utilised. Firstly, coding scheme was created by coding the expressions that were meaningful and then subcategories were constructed by combining the codes and inductive categories were formed by combining subcategories.

Considerations for content design in flipped classroom
For flipping a course, design of content refers to physical space, informing students, mindset of students, motivation, scaffolding, and chucking of course content. A teacher who is planning to flip his/her courses needs to handle each of these design steps one by one attentively, keeping tenets of flipped classroom in mind (Biggs & Tang, 2007). Quite a few studies point out the significance of physical space in implementing flipped classroom. That is to say, in flipped classroom, providing active learning environment for students means arrangement of physical classroom space which is regarded as the key to solid implementation of flipped classroom since the design of classroom space may have a considerable impact on learning of students (Carpenter et al., 2015;Fickes, 2013;David et al., 2008;Lei, 2010;Steen-Utheim & Foldnes, 2018;Swart, 2017). In reconsidering classroom space for flipping a course, forward-facing and teacherfocused formation of a classroom setting is not sufficient to encourage active and collaborative learning process. Classroom space needs to bear some characteristics that serve different academic functions by facilitating varied learning and teaching styles in a more flexible way. Fickes (2013) posits that the most critical feature of the flipped classroom design is flexibility since flipping physical spaces accentuates the existence of wheeled chairs, round tables with ample spaces around each table that students and the instructor can freely circulate. Fickes also adds that nearly half of the population (40%) learns best in informal settings. Eliminating rows where students are seated side by side in traditional classroom spaces increases spontaneous collaborative learning (David et al., 2008). An example of a basic flipped classroom is to provide separate working areas where each group or team is able to pay a visit to other groups or teams in the course of performing group or team-based tasks or activities. These findings demonstrate similar results in accordance with other research results (Carvalho & McCandless, 2014;FLN, 2014;Hamdan et al., 2013).
Reviewed studies also reveal that it is of great importance to inform students about goals of a flipped course and reasons why they need to partake in such a flipped course (Apedoe et al., 2017;Gardner, 2017;Hsieh, 2017;Van der Meer et al., 2015;Strayer, 2007). Informing students about course goals prompts them to understand what they are to complete and know before in-class time and why that knowledge is of importance. In the research comparing flipped and traditional classrooms, Strayer (2007) underlines that although students enjoy taking part in collaborative learning activities in the classroom, they are less satisfied with the design of course content with unspecified goals, thereby paving the way for a sense of unsettledness for them. In a similar vein, in order to reduce students' resistance and increase the likelihood of their willingness to adopt flipped learning as a new way of learning students must be transparently informed about how a flipped course will be implemented ( Van der Meer et al., 2015). Preparing students for a flipped course also means explaining reasons for their pre-class and in-class positions in the process (Gardner, 2017). For example, before flipping a reading course, the teacher is expected to exhibit self-regulated learning habits including the use of emotional and cognitive responses to a reading text by giving students the chance of thinking their own positions (Apedoe et al., 2017). The issue on informing students reported in the current study is in line with the finding of Hamdan et al. (2013) in that prior to their implementation of flipped classroom they prefer to inform students about the new structure of the course by exuding excitement in the potential of this teaching strategy in order to overcome students' resistance or reluctance towards flipping the course.
An important point reported in the reviewed studies is the mindset of students. Flipping a course requires flipping the mindset of students regarding their roles and responsibilities in learning (Arnold-Garza, 2014;Findlay-Thompson & Mombourquette, 2014;Newman et al., 2015;Nielsen, 2012:). Altering students' mindsets on gaining conceptual knowledge from teacher-centred learning approach to student-centred learning approach may take some time (Newman et al., 2015). This finding is supported by the researchers (Barkley, 2015;Mehring, 2016) in that flipping a course means drawing attention to students and their learning by way of altering their mindsets. In other words, flipping mindset implies changing the focus of class time and focusing attention on students and their learning by diverting attention from the teacher.
Another important point that is likely to make a significant contribution to the implementation of flipped classroom is to motivate students to complete pre-class activities and tasks such as watching a video or reading some texts (Hussey, 2014;Siegle, 2014;Suo & Hou, 2017) since students may have no desire to complete learning tasks and activities. Provided that students' motivation is stimulated, sustained, or enhanced by the teacher through motivational strategies, flipped classroom may lead to a great achievement on students' parts (Suo & Hou, 2017). Several prior studies show that students' attitudes, beliefs, and values have an impact on their understanding of course content.
In the reviewed studies, it is emphasised that scaffolding has a key role in arranging activities and tasks in many ways (Larcara, 2015;Pulley, 2014;Kim et al., 2014). For example, scaffolding could be provided by means of the order of activities, choices for students, or amount of adult intervention provided to students. In addition, while reconstructing curricular activities of a course to flip, scaffolding learning content helps the teacher reach course goals and bolster the process of teaching and learning (Larcara, 2015). Also, Kim et al. (2014) stress the significance of providing supervision and scaffolding that could enhance connection of pre-and in-class activities on students' part. Previous studies note the importance of scaffolding (Wood et al., 1978). Considered as a familiar metaphor in educational circles, scaffolding is used for ongoing knowledge construction (Cazden, 1983). According to Dewey (1938) and Piaget (1947), on the basis of ideas as to cognitive development, scaffolding plays a vital role in defining students' knowledge level and building upon that knowledge by introducing new material. In student-centred approaches supportive scaffolding is necessary to assist students in developing skills and attitudes to make the most of flipped classroom. Such scaffolding is vital characteristics of a good design in enhancing an environment where students are expected to have opportunities for mastering course content. A good example of scaffolding is for the teacher to inform students about why they need to take part in an activity, what the activity provides them, how they assess their learning based on the given activity, and how that activity impacts on their reality (Coy et al., 2017;Mayer, 2012;Willey & Gardner, 2015).
Besides, it is clear in some studies that chunking content in flipped classroom (Crawford & Senecal, 2017;Hsieh, 2017;Mclaughlin et al., 2014;Medina, 2008) is a practical solution for students to help remember information and recollect the information on the basis of human ability to process 7+2 pieces of new information (Miller, 1956). A chunk is a substantial unit of information collected from pieces of information, so chunking content and course materials are also a way of decreasing cognitive load the working memory has during instruction (Sweller, 1994). Memory can be improved if information pieces are classified into larger units, enabling information to be processed and recorded with a small number of chunks (Chase & Simon, 1973). In addition to this, since students can remember merely 10-20 minutes of an hour-long lecture (Mclaughlin et al., 2014;Medina, 2008), the teacher needs to break down conceptual knowledge into smaller pieces of components in the process of designing the content of pre-class activities in order to promote the retention of materials and learning of students (Hsieh, 2017). The findings of the current study also accord with the claims of the researchers in that the practice of chunking makes the content more manageable and accessible to students. In addition, the presentation of content in smaller segments helps students better focus on understanding the content without overloading cognitive comprehension and retention processes (Bane, 2014;Evmenova et al., 2011;Simonson et al., 2016).

Considerations for pedagogical design in flipped classroom
For flipping a course, pedagogical design pertains to active learning, its integration with some pedagogical approaches, and the role of the teacher.  7 Fulton, 2010 In the reviewed studies, it is noted that the pedagogical design of flipping a course is expected to make the classroom more active learning environment where instruction is based on a transition from catering knowledge for students to a cooperative and collaborative style of instruction that builds on what students gain through pre-class activities (Baepler et al., 2014;Hung, 2015;Pulley, 2014, Zappe et al., 2009. Helping students engage in pre-and in-class activities may require the employment of active learning in their gathering information, thinking, and problem solving via activities and tasks in pre-and in-class time. The use of teaching strategies in flipped classroom leads students to take part in active learning process. These results corroborate the ideas of researchers (Bonwell & Eison, 1991), who suggest that active learning is a broad term that focuses on students' engagement in instructional activities that occupy them with doing the activities and reflecting on what they are doing. These activity types that could be used in an active learning classroom are individual activities, paired activities, small groups, and cooperative student projects (Zayapragassarazan & Kumar, 2012). This review shows that students' learning styles and multiple intelligence are among the pedagogical considerations the teacher may bear in mind while making an effort to flip a course (Capone et al., 2017;Jones & Davis, 2008;Silver et al., 2000) since they have positive effects on students' learning process. In flipped classroom, the teacher is expected to boost students' understanding of the content and engagement in tasks and activities, taking learning styles into consideration and providing choices in terms of variety of tasks and activities that touch upon multiple intelligence and preferences. For example, in a study discussing the use of blended learning in high school classroom, all the activities are designed based on students' learning styles and multiple intelligence with a view to teaching everyday life English topics and foundational literacy (Capone et al., 2017). This study concludes that such a design of curricular activities enhances students' interest and provides them with learning autonomy enabling them to learn at their own pace. This finding is also reported in the research that students prefer one learning delivery mode over another owing to the differences in their learning styles, thus the teacher needs to take these issues into consideration while designing course content (Little, 2015). As well, according to Grasha (1996), learning styles influence students' ability to obtain knowledge and interact with peers and the teacher while they are engaged in learning activities. Thus, effective teaching is to cater for students' differences in planning the content of flipped classroom (Kyriacou, 2009;Tomlinson, 2005;. The reviewed studies also indicate that in a flipped classroom environment, students are supposed to be active participants of constructing new information through the creation of instances or situations where learning occurs through a blend of interactive and collaborative activity that involves active interchange of ideas between students within groups that perform activities (Cresap, 2017;Entezari & Javdan, 2016;Fulton, 2012). As a pre-class activity when students watch the video, pause, rewind, or replay parts of it, there occurs a kind of limited two-way interaction between students and the video, although all the interaction types may not be present in all learning occasions. As for the interaction with the teacher, the teacher is responsible for handling group discussions of students and responding to questions of students properly to help them be active learners of flipped classroom. During an activity or a discussion in in-class time, the teacher must closely observe the whole classroom by walking around the tables and listening to the discussions between group or team members in order to clarify possible misconceptions students may encounter (Fulton, 2012). As students do in-class activities or tasks, there may occur interactions between the teacher and students. As an example of interaction and collaboration in a flipped course to reveal effectiveness of active learning, students are given group-based activities and are then divided in groups to work on the given activity as a team. The teacher, at the same time, circulates around the room, observe students' discussions, respond questions, and provide feedback (Entezari & Javdan, 2016). The finding is in accordance with the results of other studies in that interaction, which is defined as a mutual action, is discreetly classified into three as interaction with materials, interaction with the teacher, and interaction between students. Interaction process is a two-way process enabling information to flow back and forth between students, teachers, and other people (Caladine, 2008). The teacher needs to create an environment where the teacher incorporates students' input by allowing them to share their experiences in an interactive and collaborative manner. By encouraging students to learn from each other, the teacher also needs to allow them to form groups where all students have the chance to respect and appreciate the contributions made by any group member of the classroom (Tinzmann et al., 1990).
Reviewed studies also underline the point that integration of flipped classroom with brainbased learning may help students participate in authentic learning experiences (Apedoe et al., 2017;Younger & Orozco, 2018) since authentic learning is also a hallmark of brain-based learning. Such integration also prompts the teacher to individualise teaching to students and their unique brains. As an example of implementation of brain-based learning in an undergraduate STEM course, the design of the course reinforces the intellectual and emotional engagement necessary to learn, hence information about bright colours and geometric shapes regarding the course content is included in learning modules to help students conjure associative memories and stimulate memory-building emotions. In that course students aiming to develop their own way of learning with the help of learning activities are empowered to read materials to recognise an order of operations for reading and understanding different types of sources (Younger & Orozco, 2018). This also accords with the notion that in deploying brain-based learning to create pre-class and in-class materials for a flipped course, meaningful connections and challenges in brain-based learning are of great importance to students (Caine & Caine, 2006;Cercone, 2006). It is therefore emphasised that collaborative learning activities that induce students to activate their unique brain-based learning process are of necessity.
Reviewed studies also indicate that flipped classroom is conducive to mastery learning by providing feedback in the form of formative assessment instruments (Apedoe et al., 2017;Prince, 2004;Pulley, 2014). The cornerstone of mastery learning lies in its view that learning is not an outcome but a process in which feedback fundamentally takes the form of formative assessment that helps students track their learning progress. To illustrate, in a psychology course, before coming to the classroom students are asked to take a short online comprehension quiz and then they are given immediate feedback as to their performances (Apedoe et al., 2017). Similarly, in a flipped instructional design course, students are asked to submit their weekly portion of projects and following their first submissions, they are asked to have group discussion and then they receive feedback from the teacher and their peers, which allows for a greater understanding of conceptual knowledge on a weekly basis on students' parts . In accordance with the present results, previous studies demonstrate that the teacher who prefers to use mastery learning must support students in giving feedback about their learning progress by using formative assessment instruments. Used as diagnostic and prescriptive purposes, feedback strengthens exactly what students are supposed to learn, assists in finding out what they learn well, and outlines what they need to learn (Guskey, 2005). Within this context, using mastery learning produces positive impacts on students' learning (Casselman, 2019;Guskey, 2007;Kreiner, 2006).
As with the use of such approaches as brain-based learning and mastery learning in flipped classroom, it is revealed that project-based learning is an alternative pedagogy to teaching students by engaging them in authentic and real-life problems or cases (Williams et al., 2014;Sams & Bergmann, 2013;Stewart, 2012;Steen-Utheim & Foldnes, 2018). In a project-based classroom, students are asked to investigate the given questions, propose assumptions and explanations, argue for their ideas, and challenge the ideas of others by trying out new ones. Drawing on constructivist theory and sociocultural theories, problem-based learning environment presents problems designed to situate learning in real-life contexts where students identify problems and construct new knowledge by activating their prior knowledge (Kilpatrick, 1918). This study supports evidence from previous studies (David 2008;Geier et al., 2008;Harada et al. 2008;Krajcik et al., 1994;Krajcik & Czerniak, 2013;Marx et al., 2004;Rivet & Krajcik, 2004;Thomas 2000). Among the key features of projectbased learning, a driving question or a problem to be solved centres upon this approach where activities and contexts in line with learning goals need to be developed by the teacher in ways that students can participate in problem solving, engage in collaborative activities to find solutions to the driving question, and finally create a set of real solutions to that driving question (Krajcik et al., 1994;Krajcik & Czerniak, 2013). Accordingly, students need to be directed to explore issues, themes or problems in-depth without pre-defined answers in stimulating and real-life learning processes (David 2008;Harada et al. 2008;Thomas 2000). In addition, a group of researchers contend that students partaking in project-based learning environment accomplish better learning outcomes than counterparts in traditional classrooms (Geier et al., 2008;Marx et al., 2004;Rivet & Krajcik, 2004).
The teacher is reported in the reviewed studies to be the facilitator who takes the responsibility of designing materials, discussing with students, and assessing students' work. In other words, the teacher plays the role of a facilitator not the deliverer of knowledge to guide active learning of students (Crawford & Senecal, 2017;Fulton, 2010;Hosler, 2017;Williams et al., 2014;Rosen et al., 2017;Steen-Utheim & Foldnes, 2018;Stewart, 2012). For instance, in a two-semester long flipped mathematics course on the investigation of student engagement, students are needed to solve problems individually and are then expected to discuss the same problems in fixed groups. The problems are handed out on a sheet of paper at the beginning of in-class session. The groups are then composed of 5-8 students to find common answers based on discussion and group learning. At the end of each session, the teacher as a facilitator spends 25 minutes going through the problems and presenting the solutions (Steen-Utheim & Foldnes, 2018). This finding is also reported by Kavanagh et al. (2017) in that the teacher as the facilitator provides sources of knowledge and assistance to help students accomplish their objectives of the course content.

Considerations for material/activity design in flipped classroom
For flipping a course, material design relates to hands-on activities, incremental material design, video features, lecture notes, incentive for reading, and activities for higher-order thinking skills. The aim of flipping a course is to make the process of teaching and learning more studentcentred and provide more classroom time to involve students in meaningful hands-on activities. Flipped learning is not limited to the use of recorded videos out of the classroom and completing homework in in-class time (Bergmann & Sams, 2012). Some teachers hold the belief in a mistaken way that videos are everything when it comes to flipping a course. What actually matters in flipping a course is not the technology but the change of instructional models that are fruitful to help students gain more instructional support (Roscorla, 2011). Reviewed studies stress that it is of great importance to design meaningful hands-on class activities and tasks requiring higher order thinking skills based on students' conceptual knowledge gains through pre-class activities (Alshehri, 2016;Bergmann & Sams, 2012;Black et al., 2017;Hosler, 2017;Kay et al., 2019;Priyaadharshini & Vinayaga Sundaram, 2018;Roscorla, 2011). In a study carried out with undergraduate students attending a computer course, students express that hands-on activities are of great benefit for them and most of the positive comments focus on hands-on activities that facilitate their understanding of programming and most importantly, those activities help them stay cognitively focused in active learning process (Kay et al., 2019). In the same way, in an engineering course at tertiary level, while designing hands-on activities the researchers employ a brainstorming activity called mind mapping to make students active in the process and measure their skills of critical thinking and problem-solving (Priyaadharshini & Vinayaga Sundaram, 2018).
The current review highlights that discussion, argument, brainstorming, and debate are among common types of hands-on activities (Alshehri, 2016;Black et al., 2017;Hosler, 2017). With the implementation of active learning approach in flipped classroom, there are numerous in-class collaboration techniques that include but not limited to an open discussion, argument, and brainstorming (Hosler, 2017) that lead students to partake in activities in a more meaningful and engaging manner. For example, an open discussion or an argument is a manageable and effective technique with any size of student groups in flipped classroom. Depending on the topics and comments of students in discussions or arguments, students are supposed to think, explore, evaluate and construct new ideas. As for brainstorming, which may be conducted face-to-face or online in a flipped course, it is a viable solution for in-class active thinking and learning process of students who are expected to discuss and explore solutions to the topics (Alshehri, 2016). Another way of designing materials that engage students in in-class time is a debate activity where students could be for or against the given subject personally or in groups through a very systematic and rational manner. The use of this technique not only improves students' verbal communication skills but also builds confidence in themselves. For example, in the research probing the use of flipped classroom, it is highlighted that when lectures and assessment are conducted out of the classroom, students start to engage more meaningfully in course content and even the quietest students in the classroom start talking and have something to say thanks to such debate activities (Black et al., 2017). These results are in accord with recent studies (Demetry, 2010;Maher et al., 2015;Warter-Perez & Dong, 2012;Crews & Butterfield, 2014) in that since flipped classroom provides ample time to make room for meaningful and interactive tasks and activities, devising such hands-on learning activities that promote active learning leads to more effective learning. In addition, in an extensive study carried out with nearly 95,000 graduates, Scott (2005) contends that a repertoire of interactive learning strategies such as group discussions and debates are appreciated by those graduates. Small heterogenous groups of learning communities including 3-7 students could be formed to achieve an activity or a task in preclass or in-class time in an effort to give a presentation, write a report, or criticise a reading text led by group leaders in each group.
Besides, reviewed studies show that the design of class materials needs to have less openended and bear incremental design characteristics to reflect on students' own learning and students are otherwise likely to be reluctant to complete pre-and in-class activities Strayer, 2007;Toto & Nguyen, 2009). These findings seem to be in line with recent research that in blended learning settings, incremental design principles can be applied in the process of developing course materials (Hinkelman, 2018;Ogden & Shambaugh, 2016). In a flipped algebra course, for example, students are asked whether they recommend the video to a friend who has difficulty in understanding the topic, and 93% of fifty-five students respond to the question positively since the video follows clear and step-by-step procedure about the presentation of content (Ogden & Shambaugh, 2016).
Furthermore, reviewed studies underline the fact that the concept of chunking needs to be carefully dealt with in making a video for fostering conceptual knowledge (Burton, 2013;Faulkner & Green, 2017;November & Mull, 2012;Van der Meer et al., 2015;Prince, 2004;Toto & Nguyen, 2009). Since the video is regarded as one of the key conceptual learning moments with a view to help students get ready for dynamic engaging in-class discussions (Prince, 2004), it needs to have some degree of interaction and be delivered in shorter and concise chunks of knowledge Faulkner & Green, 2017). In a study carried out by Van der Meer et al. (2015), it is highlighted that the average length of the video should be under 15 minutes maximum. Burton (2013) also states that videos must be short, no more than 6-7 minutes in length, that is, the shorter and concise video without errors and pauses is considered to be much better since students are easily distracted while watching the videos (Toto & Nguyen, 2009). These findings corroborate the ideas of researchers as to how to make videos for flipped classroom (Engin, 2014;Chen et al., 2014;Lee & Wallace, 2018;Raths, 2014).
Together with the use of videos, reading texts play a leading role in delivering pre-class content in flipped classroom (Apedoe et al., 2017;Berrett, 2012;Black et al., 2017;Cresap, 2017;Sherr et al., 2019;Swart & MacLeod, 2016). According to an informal survey concerning the use of textbooks, more than half of the students do not read the reading texts, share the textbook with another student, and just have a quick look at it after class or before a test (Cresap, 2017). Students verbalise that they have difficulty in understanding reading texts of the textbooks because of their length. Textbook readings are generally lack of engagement. Lecture notes are thus considered as the anthesis of a textbook reading. In a study, most of the students prefer the lecture notes called "Socratic Lecture Notes" over textbooks (Swart & MacLeod, 2016) since they are provided with summaries of the texts in order to foster and enhance their conceptual understanding, increase their in-class participation, and perceived usefulness (Apedoe et al., 2017). These results match those observed in earlier studies that use of reading texts in flipped courses shows only a small percentage of students (18-33.9%) read the assigned reading texts before coming to in-class session (Berry et al., 2011;Burchfield & Sappington, 2000;Stelzer et al., 2009).
The low rate of students' completing reading tasks may be attributed to different reasons. In essence, ensuring that students read the assigned texts requires some incentives (Berrett, 2012;Black et al., 2017;Brown, 2018;Sherr et al., 2019) which could be utilised in a number of ways in order to help students take part in higher order thinking skill activities with the solid conceptual knowledge at hand and otherwise may decrease the perceived usefulness of flipped classroom. For instance, students could be asked firstly to read the assigned texts and take an online comprehension quiz before in-class time to check whether they complete the assigned readings and acquire conceptual knowledge. The quiz questions should aim at clarifying and extending the points in the readings (Berrett, 2012). Accordingly, quiz results allow the teacher to expand on the points where students have difficulties (Black et al., 2017). This type of quiz can be used as a method of holding students accountable for reading texts out of the classroom or class may begin with a formative assessment instrument in the form of a quiz for pre-class content (Sherr et al., 2019). Since the aim of designing pre-class activities is to prepare students for in-class time and check their understanding of conceptual knowledge through reading texts, students may alternatively be asked to complete an online survey about the content prior to in-class time to lead an in-class activity or they may be required to take a quiz covering the questions with moderate levels of difficulty about the content to check their pre-class learning (Brown, 2018). The findings of the current study further support the ideas of Brame (2016) in that there are a number of incentives for students to get prepared for in-class time. These range from online quizzes to worksheets to short writing assignments.
Reviewed studies also stress the importance of designing activities that prompt students to use higher-order thinking skills in flipped classroom (Alshehri, 2016;Black et al., 2017;Estes et al., 2014;Heo & CHun, 2018;Hosler, 2017;Overmyer & Dennis, 2016;Rosen et al., 2017), hence Bloom's Taxonomy plays an important part in designing pre-and in-class time activities so that students may establish higher order thinking skills through properly designed in-class activities demanding lowerorder thinking skills to construct conceptual knowledge solidly. Doing so deepens students' learning and leads to retention of the knowledge and reinforcement of critical thinking skills (Overmyer & Dennis, 2016). In a study questioning the impact of flipped classroom on language learning, the researchers design the content of the course in line with the principles of Bloom's Taxonomy by moving lower-order thinking skills part of the course out of classroom and inserting language activities at the level of analysis and creation into in-class time (Rosen et al., 2017). By the same token, a study investigating the effects of using flipped classroom on improving higher order thinking skills concludes that after watching pre-class videos and taking part in activities designed in line with Bloom's Taxonomy, students attending a STEM course start to ask better questions and try to make the right questions. As learning progresses, the proportion of questions in testing higher order thinking skills increases noticeably (Heo & CHun, 2018). These results are reflected in the recent studies of a group of researchers (Lee & Lai, 2017;Lockwood, 2014) who also propose that design and use of activities in line with Bloom's Taxonomy are feasible to promote students' higher-order thinking skills (Lee & Lai, 2017;Lockwood, 2014).

Considerations for delivery design in flipped classroom
For flipping a course, design of content delivery refers to use of a learning management system (LMS) and tools for material creations.

Table 5. Considerations for content delivery design
Sub-categories f Sample research Use of an LMS 7 Vassiliou, 2013 Tools for material creation 11 Carstens & Sheehan, 2015 Since flipped classrooms bears some characteristics of blended learning, provision of course content could be made via learning management systems. Learning management systems (LMSs) such as Moodle, Blackboard, Desire2Learn, Canvas and many more are rich in providing tools for designing activity and assessment that may support teaching and learning in flipped classroom (Burke & Fedorek, 2017;Crawford & Senecal, 2017;Murillo-Zamorano et al., 2019;Panuwatwanich, 2017;Priyaadharshini & Vinayaga Sundaram, 2018;Rosen et al., 2017;Vassiliou, 2013). Although fundamentally utilised to provide such pre-and in-class content as videos, podcast, narrated PowerPoint, or reading texts, an LMS also provides a platform where the teacher can guide students to actively engage in some activities (Crawford & Senecal, 2017). Furthermore, the use of an LMS for the delivery of both pre-class materials and in-class activities or tasks provides synchronous engagement and interaction for students having different learning styles. As well, using an LMS in flipped classroom provides the teacher with the flexibility in designing or adapting different types of quizzes or exams in order to ensure students' success in flipped classroom (Vassiliou, 2013). For instance, in an undergraduate flipped course regarding crime control, the researcher prefers Moodle as the learning management system since it is available to use for students at university, and the course content including stories, YouTube clips, the syllabus, and assignment guides are all uploaded to the LMS (Burke & Fedorek, 2017). In the same manner, in an undergraduate flipped engineering course, the course contents prepared by the teacher in various formats that appeal to learning styles of students are provided to students through the use of an LMS. In addition to this, the embedded tools and modules in the LMS such as mind mapping are utilised in order to evaluate students' competencies about the course content (Priyaadharshini & Vinayaga Sundaram, 2018). In a recent study assessing the effect of flipped courses on Russian and Japanese students' speaking skills, through an LMS, students are provided to watch videos and their grammatical explanations and they are then required to take an online conceptual knowledge check in the LMS before coming to class. Meantime, they have the chance of taking this check with automatic feedback 3-5 times. The feedback from the check also leads students to the identification of any misconception about grammar topics (Rosen et al., 2017). In a similar way, in flipped macroeconomics and postgraduate engineering courses students are able to watch and summarise videos and then answer online questionnaires via an LMS. After completing their summary submission through the LMS, they are able to participate in other activities. They have for example the chance of taking online quizzes, engaging in online discussion board for question and answer, and getting support from virtual meeting sessions (Murillo-Zamorano et al., 2019;Panuwatwanich, 2017). The findings of the current study are in complete agreement with the ideas of the researchers as to the employment of learning management systems for content delivery (Cupak, 2018;Elmaadaway, 2018;Gaughan, 2014;Zainuddin & Perera, 2018).
Reviewed studies also note that apart from the use of LMS platforms for the provision of the course content, creation of pre-and in-class materials requires use of various applications and tools both free and paid (Carstens & Sheehan, 2015;Crawford & Senecal, 2017;Gardner, 2017;Kotlik, 2017;McBride, 2015;McCarthy, 2016;Price & Walker, 2019;Raftery, 2011;Schrlau et al., 2016;Springen, 2013). The teacher can benefit from common software for recording course materials such as PowerPoint, Camtasia, Snagit, Tellagami, Pow Toon, and Adobe Presenter (Gardner, 2017;Price & Walker, 2019). When developing PowerPoint presentations, audio record can be inserted into a slide presentation so that students can review audio lecture and visual elements synchronously (Crawford & Senecal, 2017). For example, in a flipped course of 9 th grade global history and geography, the teacher modifies PowerPoint slides by adding a voiceover that explains the slides and more notes to them for fear that students cannot understand general bullet points of information. As a result of the modification process of the slides, the old lecture notes are turned into guided notes with pictures and most of the students (75-80%) express that with the help of the guided notes they could complete their assignments and reinforce their knowledge they acquire from those notes (Carstens & Sheehan, 2015). In Springen's (2013) study, students express their satisfaction about the use of screencast in flipped classroom since it gives them the flexibility of pausing and rewinding while taking notes. They also mention it is helpful for them to have each chapter broken up into a couple of screencasts lasting 10 to 15 minutes. A short and sharply focused screencast could be beneficial to supporting students at their own pace to attain learning goals of the course (Raftery, 2011). In a recent piece of research , the researchers make use of Tellagami (free) and Pow Toon (free) applications to create resources for the course. In addition, they create a video from narrated PowerPoint slides by using tools like PowerPoint (paid), Camtasia Studio (paid), and a webcam. These tools also enable them to insert quiz questions into videos. Similarly, in a case study with first-year history students, the researcher uses Snagit to produce an image capture for a simple screenshot or a video capture. After screen recordings are saved by the researcher, they are uploaded to YouTube (Kotlik, 2017). For flipping a course, the researchers employ Adobe Presenter, with which slide casts, or brief narrated, and animated slides are created in 15-minute length for students to understand pre-class course content. Livescrib Smartpen technology is also utilised to create pencasts that are called narrated written notes that provide students with content for each week's topic in the format of downloadable interactive PDF. In doing so, students are able to have opportunities such as printing them, playing them, clicking backward or forward on them, adding notes to them, and rewriting notes on them in their own style (Schrlau et al., 2016). These findings support the ideas of the researchers regarding the use of tools for material creation (DeRuisseau, 2016;Gardner, 2012;Gaughan, 2014;Stanley & Lynch-Caris, 2014).

Considerations for assessment design in flipped classroom
For flipping a course, design of assessment involves formative assessment instruments, taxonomy-based questions, quiz as an incentive, and the use of an LMS for quizzes.  6 Crawford & Senecal, 2017 In the reviewed studies, it is identified that implementation of flipped classroom helps the teacher to fine-tune assessments to the course goals by using formative assessment instruments (Apedoe et al., 2017;Bergmann & Sams, 2014;He et al., 2018;Hosler, 2017;Jeffries & Huggett, 2014;Stowe, 2010;Strygacz & Sthub, 2018), thereby minimising failure in terms of summative evaluation in a traditional classroom. Also, flipped classroom presents the opportunities for the teacher to observe the progress of students formatively while they are engaging in in-class activities and enables the teacher to address any misconception if needed (Bergmann & Sams, 2014). Moreover, formative assessment in flipped classroom may be a mix of group and individual assessments. Group assessment could be conducted for the development and facilitation of in-class discussion and activities (Apedoe et al., 2017). In flipped classroom, there are a number of formative assessment tools from which the teacher can benefit when they are in need of assessing students' progress. The use of entry ticket is one of the most preferred assessment instruments for measuring students' readiness. To illustrate, in a study on simulation-based training in flipped classroom where students are supposed to take part in activities requiring higher order thinking skills, students are initially asked to take an entry ticket quiz to ensure that they are ready for those problem-solving activities (He et al., 2018;Strygacz & Sthub, 2018). Additionally, with one-minute paper, the teacher can assess students' prior knowledge and check their understanding of key concepts through one or two questions (Stowe, 2010) that may require students to summarise some key points of both pre-class materials and in-class discussions. In accordance with the present results, previous studies demonstrate that since flipped classroom has active, constructive, and student-centred learning characteristics, formative assessment with various instrument types is more practical to understand how much progress students display towards course goals and it also allows the teacher to continuously monitor students to make adjustments if needed (Akkaraju, 2016;Chen et al., 2014).
Reviewed studies also highlight that apart from its use in designing activities and materials for flipping a course, Bloom's Taxonomy contributes significantly to designing questions that demand lower order and higher-order thinking skills (Adams et al., 2016;He et al., 2019). For instance, in a flipped genetics and evolution course, the researchers assert that cognitively demanding assessment has a profound impact on the effectiveness of active learning. Each exam is thus designed in a way that represents 50-73% of the questions in level 3 (Apply) and only 17-24% are in level 1 (Remember) so as to check students' mastery level of conceptual knowledge and their application of that knowledge (Adams et al., 2016). Similarly, in an undergraduate flipped chemistry course, each of the videos is accompanied by an assignment that includes remembering and understanding level questions regarding the conceptual knowledge in videos (He et al., 2019). When formative assessment instruments for flipped classroom is well developed in accord with course goals and tenets of Bloom's Taxonomy, they function as perfect learning and assessment opportunities for students and the teacher. These results support evidence from the ideas of the researchers (Jensen et al., 2015;Sarawagi, 2014) in that besides its being a guide to classify and order the content of the course, Bloom's Taxonomy serves as a pedagogical framework to help the teacher prepare questions for lower and higher-order thinking skills in flipped classroom.
Reviewed studies also stress that use of a quiz is reported to be an incentive for students (Enfield, 2013;Gikandi et al., 2011;Hsieh, 2017;Kenney & Newcombe, 2014;Kotlik, 2017;Rosen et al., 2017;Swart & Macleod, 2016). According to Enfield (2013), a quiz can be used as an incentive to ensure what students must do prior to in-class time. For instance, in a flipped history course some students feel that there is little motivation for them to watch videos since it is not compulsory. It is therefore underlined that randomly selected short online quizzes including five questions regarding assigned videos could work well as an incentive to hold students accountable for watching them and doing pre-class activities (Kotlik, 2017). In some recent studies questioning the effectiveness of flipped classroom, majority of the students express that online conceptual knowledge check is beneficial because of its immediate feedback and simplicity. They also underline that it helps them to check whether they have understood what they are learning especially before taking the exam (Hsieh, 2017;Rosen et al., 2017). Similarly, in a flipped educational psychology course, students are informed that they are responsible for reading the textbook, listening to narrated PowerPoint lectures, and viewing video clips. Students are then asked to take an online quiz the night before in-class time. The use of online quiz allows the teacher to check their understanding of the materials and identify what needs to be reviewed in in-class time by giving students the chance of seeing which areas need more careful study (Kenney & Newcombe, 2014). In addition, if students' retake of an online quiz is also designed in flipping a course, it may open up an opportunity for students to understand whether they are ready for the following course goals and it may additionally save enough time for in-class time to deal with any possible problems. Students who have the chance of retaking the online quiz appreciate such kind of assessment as being effective in helping them grasp course content (Gikandi et al., 2011;Swart & Macleod, 2016). The findings of the current study are in agreement with recent studies (Kaw et al., 2019;Swithenbank & DeNucci, 2014) indicating that as active learning participants of flipped classroom, students may be required to take an online quiz which enables the teacher to judge what subjects they have difficulty in understanding prior to in-class time.
Reviewed studies accentuate that an LMS with its affordances of such question forms as multiple choice, true/false, matching, ordering, fill-in-the-blank can be used to watch and assess students' learning progress in flipped classroom (Crawford & Senecal, 2017;Hsieh, 2017;Kay et al., 2019;Shinaberger, 2017;Swart, 2017). Thanks to the features of automatic grading and feedback response, an LMS assists the teacher in obtaining the analysis of quiz results, allowing the teacher to check whether students need any correction or remediation (Crawford & Senecal, 2017;Shinaberger, 2017). For example, in a study conducted with undergraduate business students, following interactive group learning activity (IGL), each group member takes an individual online quiz with a password and it takes about 20 minutes or less to complete and then each of them is graded by the learning management system (Swart, 2017). Similarly, in a flipped computer programming course at a college, students are asked to take a quiz including randomly selected 15 multiple-choice questions from a database of 550 questions and are expected to answer the questions within 10 minutes to be assessed about their conceptual knowledge and comprehension of each unit's content (Kay et al., 2019). In the same vein, in a flipped curriculum, instruction, assessment and classroom management course, pre-class assessment that counts for about 10-15% of the total points of the course content is designed to measure conceptual knowledge of core course goals. Students are asked to take the quiz due a day prior to class. Following the completion of in-class activity or task, an individual quiz that is completed and graded through the LMS shows if each student has accomplished the learning goals of that session (Hsieh, 2017). These findings are consistent with the literature in that designing online quizzes is supported by many of learning management systems and assessment modules in them to help the teacher acquire the results after the system marks quizzes automatically (Awidi & Paynter, 2019;Karlsson & Janson, 2016).

Conclusion and Recommendations
Flipping a course means more than shifting the format of a course, that is to say, it entails the replacement of traditionally designed classroom with a flexible learning environment where students have better performances through the existence of working stations with wheeled chairs and tables. More importantly, flipping a course is to help students alter their mindsets about the new form of learning to hold them accountable for learning pre-class content and willing to work with their peers and the teacher in in-class activities (Banas Velez-Solic, 2013;Cresap, 2017).
Although flipping is suitable for some courses, flipped classroom may not lend itself to every course, teacher, or student (Beck & Ferdig, 2008;Osguthorpe & Graham, 2003;Strayer, 2012). However, choosing a course to flip is one of the most significant decisions, hence students' expectations and needs should be identified by the teacher prior to the process of designing a flipped course (Foertsch et al., 2002;Groves & O'Donoghue, 2009;Heinze & Procter, 2006). Besides, designing a flipped course necessitates more advanced instructional design planning and additional time to prepare course materials. The teacher must therefore plan for, chunk, look for or record pre-class materials, integrate pedagogical approaches with collaborative learning strategies, and monitor students' understanding based on formative assessment instruments. Since the objective of educational technology is to affect and effect learning of students in today's classroom, the teacher should benefit from software and applications to create necessary course content. Students must be provided with a range of learning activities from simpler types of activities to more complex ones in order to reinforce and deepen their understanding of course content. As well, learning means making connections, so connections or associations are of great importance as they provide patterning for students to participate actively in learning process, thus as Weiss (2000) suggests, the teacher should create an environment where fun and meaningful learning situations and challenges to students' thinking skills are ensured.
Though considered as the novelty in the field of education, flipped classroom bears some characteristics of such pedagogical approaches as mastery learning, active learning, brain-based learning, problem-based learning, and multiple intelligence. To the best of my knowledge, what flipped classroom differs is that it might have the potential to combine best of all the knowledge we have with technology of the 21 st century to make students well equipped with necessary knowledge and skills in their learning process to be global competitors of future. Hence, if the teacher takes some considerations such as content, pedagogy, material, delivery, and assessment designs account attentively, design of flipped course that support students in both acquiring conceptual knowledge and enhancing higher order thinking skills may produce better performance for students.