Investigating student engagement with intentional content: An exploratory study of instructional videos

in


Introduction
In recent years, flipped classrooms have become increasingly popular in higher education environments, conceived as an opportunity to help students reach higher learning levels and goals (Bergman & Sams, 2012;Strayer, 2012;Walsh et al., 2019;Walsh & Risquez, 2020;Wang, 2017). In tandem, there is a growing interest in student engagement in higher education (Healey et al., 2014;Higher Education Authority, 2016;Kahn, 2017;O'Brien et al., 2019;Walsh et al., 2019). Student engagement is considered a prerequisite for learning (Fredericks et al., 2004;Guo et al., 2014) and is generally understood as a multi-component construct that includes behavioural engagement which relates to participation and involvement in learning activities (Fredericks et al., 2004).
Given this importance on engagement, there is also now an increasing emphasis on engagement analytics and educational data mining to identify how students directly engage with content and resources. Perhaps such trends are attributable to the continuous quest to transition from traditional teaching methods to modern methods. Traditional methods, in the main, are instructor-centred, whereby the lecturer assumes the role of a 'sage on the stage' (Armstrong and Fukami, 2009). The sage teaches, disseminates instructions, explains the items; meanwhile the student audience is passive, standing still and are just expected to learn (Deslauriers et al., 2019). Conversely, with modern teaching methods, lecturers rely on the student, placing them as learners at the core. Students are the real actors in their learning process, echoing Confucius "I hear and I forget, I see and I remember, I do and I understand", thus promoting that real learning only takes place when the learner is actively and directly involved in the learning process. When this is the case students become more aware of, and responsible for their own learning which provides for improved and increased self-confidence, self-awareness, responsibility and autonomy (Bergman & Sams, 2012). One such modern approach to teaching is the FLIP approach, itself heavily rooted in socio-constructivist theories of education and active learning (Oliván Blá).
The FLIP classroom is an instructional strategy that provides a new methodology for teaching and learning, in which one-to-one interaction and a more co-operative and collaborative approach improves the teaching and learning experience (Onojah et al., 2019). The FLIP approach is defined "as a pedagogical approach in which direct instruction moves from the group learning space to the individual learning space", and fits into the broader category of blended learning (Flipped Learning Network, 2014). A flipped classroom sometimes known as an inverted classroomtypically refers to a learning environment whereby course materials are provided to students in advance of the face-to-face (f2f) session so that traditional outside of class exercises/activities can be undertaken during class time, under the direction of the instructor (European Commission, 2014;Tucker, 2012;Yarbro et al., 2014). The concept of a flipped classroom is in stark contrast to the notion of requiring students to attend face-to-face lectures in a (physical) classroom/laboratory environment so that material can be disseminated.
The FLIP classroom is often thought of as a cycle because students typically watch an (instructional) video in advance of the classroom setting, subsequently discussing and applying that knowledge in class, and then continuing on to watch another video for new content introduction, in a continuous loop. For example, instructors may record short screencast videos of their lectures or podcasts and upload them to a virtual learning environment (VLE). In some cases, massive open online course (MOOC) course materials are also used to support flipped classroom environments (Adair et al., 2014). Students then review these materials in advance with a view to being better prepared to engage more actively in class. Ideally, a flipped classroom session should focus on collaboration and interactive problem-solving activities, with the instructor providing more of a support and facilitation role (Hamdan et al., 2013). While the flipped classroom approach may initially sound pedagogically superior to the traditional one-to-many dissemination approach, the quality of course materials and instructor support is still crucial to the success of a flipped approach. The concept of flipped learning is different than a flipped classroom, as it involves more than merely flipping or inverting activities.
To engage in flipped learning effectively, instructors ideally incorporate four pillars into their educational practice (F.L.I.P.): Flexible Environment, Learning Culture, Intentional Content and Professional Educator (Hamdan et al., 2013). The flexible environment pillar refers to both the physical as well as the virtual student engagement space. It caters for both individual and group interactions. 'Flexibility' refers to the 'anytime and anywhere' access and engagement with the enhanced technology learning environment. A flexible learning environment encourages active learning, promotes collaboration, increases student-faculty interaction and overall enriches the student educational experience. Students can therefore select 'when and where' they learn. From an instructor's perspective, the expectation of student timelines for learning and assessments is flexible as long as key deadlines are met. The learning culture pillar refers to moving away from the traditional instructor-centred approach where the instructor (teacher) is the primary source of information -to a learner centred approach where in-class contact time is utilised to discuss topics in greater detail therefore creating a richer learning opportunity. As a consequence, students are "actively involved in knowledge construction as they participate in and evaluate their learning in a manner that is personally meaningful" (Flipped Learning Network, 2014).
The 'Intentional content' pillar, the primary focus of this particular paper, refers to learning material used to cultivate the student's conceptual understanding as well as practical application of concepts. The objective of intentional content is to provide the best possible student learning experience through the combination of student self-discovery and theoretical material. Facilitators or instructors, in designing intentional content give careful consideration to both the content and the presentation method of material prior to contact sessions to maximise face-to-face effectiveness. The last pillar of the F-L-I-P classroom refers to the 'Professional educator' whereby the instructor becomes the 'silent partner' and facilitator, transitioning from the 'sage on the stage' philosophy to the 'guide on the side' (Blau, 2011); a role typically far more demanding than their traditional educator role. Furthermore, the professional educator i.e. the instructor, requires a more specialised skillset than in a traditional classroom environment (Horn & Staker, 2014). By flipping the class the instructor's role changes to that of a facilitator of learning through observing and monitoring areas in which students need help with; providing students with different ways to learn content and demonstrate mastery; giving students opportunities to actively participate in meaningful learning activities; scaffolding these activities and making them accessible to all students through differentiation and feedback; and conducting ongoing formative assessments during class time (Karabulut-Ilgu et al., 2018).
The Flipped Learning Network claims that the third pillar of F-L-I-P i.e. Intentional Content is one of the most neglected pillars of flipped learning (www.flippledlearning.org). Thus this paper focuses primarily on (outside the classroom) Intentional Content, provided to maximise (face-to-face) classroom time, to better utilise teaching and learning methods for a more student-centred, active learning experience. Intentional content is all about choosing the best content to be delivered in the classroom, and the best content to be delivered outside of the classroom. To this end, the exploratory study herein uses log-file data generated by the Learning Management System (LMS) to identify patterns, usage, and engagement of 468 (first year) undergraduate Bachelor of Business Studies (BBS) students at the University of Limerick, placing particular emphasis on the use of instructional videos. Firstly, the study investigates students usage and engagement with instructional videos as intentional content and how they are used as a foundation for learning in a FLIP classroom. Secondly, the study investigates usage frequencies and specific intervals of engagement, with a view to identifying critical times in the semester where intentional content (i.e. videos and supporting material) are perceived as essential by students. Thirdly, the study explores how intentional content (i.e. instructional videos) supplements other elements of the FLIP environment.

Instructional videos as intentional content
Intentional content refers to the instructor's informed, good judgement about what content needs to be taught directly, and how to Table 1 F-L-I-P framework applied.

F1
I establish spaces and time frames that permit students to interact and reflect on their learning as needed.
The instructor tasked students to conduct activities one week in advance of face-to-face lab sessions by performing online preparatory tasks. The learning process was further substantiated by online discussions using a Virtual Learning Environment (VLE), online instructional videos and discussions during class contact sessions. Solutions posted online at the end of the week for students to reflect on their learning.

F2
I continually observe and monitor students to make adjustments as appropriate.
The feedback received from the VLE was a useful indicator of the level of engagement of the preparatory tasks student were required to do in advance of class contact sessions. This was used to prepare/revise material for in class contact lab sessions. Also. attendance was recorded at all weekly lab sessions and students were encouraged to upload their completed solutions for further feedback.

F3
I provide students with different ways to learn content and demonstrate mastery.

Online activities, class-contact sessions, a mid-term practical exam, along with a final (practical) exam provided students with various ways to demonstrate that the required level of knowledge and skills had been achieved. L
Learning Culture

L1
I give students opportunities to engage in meaningful activities without the teacher being central.
Problem based learning exercises (along with solutions) were provided for students to gain mastery of the respective skillset. Students were able to complete these tasks either on or off campus with or without the instructor being central/present.

L2
I scaffold these activities and make them accessible to all students through differentiation and feedback. A great deal of work went into the development of the (reusable) instructional videos provided to students via the VLE one week in advance of the respective lab session. These videos were designed specifically for this particular module.

I3
I differentiate to make content accessible and relevant to all students.
Various delivery formats were used such as instructional videos and readings, lectures, and spread sheet solutions. Optional additional exercises were also made available for interested students to advance their own personal skillset on a self-directed basis. These optional exercises are based on the content presented in the instructional videos. P Professional Educator

P1
I make myself available to all students for individual, small group, and class feedback in real-time as needed.
At the start of the semester, the learning management system was used to communicate pre-arranged office hours to students. Instructors could also be contacted at any time via e-mail and additional meetings could be arranged upon request. Online discussion forums were moderated and feedback was provided where applicable. Groups also had a scheduled face-to-face 1-h tutorial session weekly. Instructors also made themselves available after class to assist with any queries. P2 I conduct ongoing formative assessments during class time through observation and by recording data to inform future instruction Two practical exams took placeone during the term and one at the end. Tutor signed-off on the work each student undertook during the lab session before same was uploaded to Sulis. A spreadsheet was continuously updated on google docs where the module team were able to observe student engagement and identify possible 'at risk' students early.

P3
I collaborate and reflect with other educators and take responsibility for transforming my practice.
The module team met regularly during the semester (module leader, teaching assistant and tutor) to address any issues that may have come to the fore and practice was revised/adjusted accordingly.
best introduce that content in their instructional video 'lectures' and supporting resources and material. According to Hamdan et al. (2013), a professional educator knows what falls within the scope of their modules, which issues are vital, the significance of each topic, and how each topic relates to the module and the learning outcomes for students. Professional educators use intentional content to maximise classroom time in order to adopt methods of student-centred active learning strategies. Having access to this intentional content for learning allows students to fully interact with learning within and outside the classroom (Olumorin, 2009). Professional instructors can offer video-based materials and other supporting learning resources for students to explore in their own time to maximise the face-to-face interactions. A variety of intentional content resources are available including text-based, audio-based and video-based communications, which can be used to modify and adopt teaching time and space (Onojah et al., 2019) and so the influence of ICT plays a significant role (Anthony, 2012). The use of videos has experienced a steady increase (Mirriahi & Vigentini, 2017). The design and selection of videos is not simply an 'add-in', instead they are a core means by which students are introduced to content via direct learning, and so they need to be selected carefully (or carefully produced) by the professional educator within an overarching curriculum for the module (Bergman & Sams, 2012). According to Hamdan et al. (2013), only key topics and critical information need to be introduced through instructional video and when used in tandem with other resources, video content provides a solid foundation knowledge for in-class activities where students apply and synthesise knowledge (Seaboyer, 2013). With recent advances in video streaming technologies, learners' digital footprints when accessing videos can be mined and analysed to better understand how they learn and engage with them. Although much research has been done, particularly focusing on psychological aspects, the educational value, and the user experience, the advancements of the technology and the emergence of analytics provide an opportunity to explore and integrate not only how videos are used in the curriculum but whether their adoption has contributed towards learner engagement or indeed the learning process itself (Giannakos et al., 2014). Kim et al. (2014) carried out a large-scale analysis of in-video drop-outs and peaks in viewership and student activity using data mined from 862 videos across four different MOOCs. They found that when videos were too long, students tended to abandon them. Points of interest and/or possible points of confusion were indicated by re-watching common sections of video (peaks). Kim et al. examined these peaks further and attempted to identify explanatory student activity patterns. Other studies have explored when and how students use online instructional videos. Schiltz (2015) used Google Analytics to examine usage patterns of video tutorials created to supplement introductory physics lectures for engineering students. Interestingly, they found that although not compulsory, most students used the video tutorials and showed a high level of engagement with the materials. While students viewed video tutorials throughout the term, a significant number of students also used them as a revision aid for exams later in the term. Metz (2013) investigated the impact of (short) assigned online videos on student learning in an introductory biology programme. By tracking access, the data showed that in a 'flipped' classroom environment, video watching was consistently above 80%, suggesting that videos work well in supporting learning outside of the classroom, freeing-up precious (face-to-face) class time. They also found that if the 'viewing burden' is heavy and students are not incentivised, videos tend not be as well received. Brady et al. (2013) measured attendance and online lecture video accesses to determine if students use online recordings of live lectures to catch-up after missing a class and also, more importantly, within what timeframe. They found students tended to use the videos variably, but when an exam is forthcoming, access patterns, particularly amongst absentees, showed a significant increase. This finding is consistent with earlier findings by Brotherton and Abowd (2004) who found a peak in access occurs around exam time. In a more recent study, Delaviz and Ramsay (2018) used data generated from YouTube analytics to determine the viewing patterns and usage of 76 short-topic videos by first year students on an introductory course in engineering. Videos were prepared with an average length of 8:11 min, each covering a specific and discrete topic from the course. Based on the YouTube analytical data, they were able to identify, for example, the number of views, the average view durations and identify the different sections of the videos that were watched multiple times.

The study setting: applying the FLIP criterion
To ensure best practice of a F-L-I-P classroom approach, all of the criterion identified in Table 1 were met in designing the practical (flipped) element of a Business Information Management module, which covers spreadsheet skills using Microsoft Excel, ensuring the correct blend of practical and academic knowledge and skills.

Intentional content of the module
Intentional content enables an instructor to offer materials for students to explore in their own time to maximise the face-to-face interactions. Business Information Management is a mandatory module all first-year undergraduate students have to undertake as part of their BBS degree at the University of Limerick. The module assumed no prior knowledge of the practical material although some students had previously used spreadsheet software. Students attended five 1-h computer labs over the autumn semester covering seminal spreadsheet topics and sat a 1-h end-of-term practical spreadsheet exam at the end of the semester (December).
A series of intentional content i.e. instructional videos, were developed specifically for this module. The videos covered five topics and demonstrated how certain tasks could be implemented using Microsoft Excel. Videos were designed closely in line with best practice (Mayer, 2014) and adhered to other guiding principles as presented in Fyfield et al. (2019) such as ensuring high audio quality, coherence (only instructional material directly related to the key learning goal be included), full worked-through examples, etc. Also, the suite of videos was unanimously approved for use by the module design team in advance of the semester (two of which incidentally have postgraduate qualifications in technology-enhanced teaching & learning in higher education). Students were expected to engage with this video-based intentional content one week in advance of the face-to-face practical session, which took place in a computer laboratory. The videos were recorded using 'CamStudio' and included an audio commentary. Videos were recorded in short/manageable chunks to ensure that students were not overwhelmed by any one topic. In total, there were 17 instructional videos with a combined total of 7,460 unique views over the semester. Comprehensive video details are provided in Table 2. It should be noted that a 'view' is similar to a 'hit' for a website; once a video is accessed, for whatever length of time, it is counted as a view.
All instructional videos were intentionally uploaded as unlisted on YouTube with links provided only through the university's virtual learning environment (VLE). This was to ensure only course participants accessed the videos. The number of hits verified that views were only logged while the course was running. In addition to the weekly instructional videos, students were also provided with the following supplementary/supporting resources: • general instructions (pdf) for the weekly lab session describing the topic being covered • an excel spreadsheet template in which to complete the weekly task • a question sheet (pdf) that required students to apply concepts covered in the instructional videos viewed one week prior to the face-to-face weekly lab session (problem-based learning). • an excel spreadsheet solution made available at the end of each week Although the videos were uploaded to YouTube, the supporting materials were uploaded to the VLE. While YouTube analytics could provide data on usage, this could not be related to individual students, nor linked to their accessing of other resources. Therefore, this study only used YouTube as a platform to host the videos. VLE log-file data was exclusively mined for this research as it identified when each student accessed each type of resource.
Face-to-face expert support was available at each of the five weekly computer labs during which students could complete the question sheet. They were encouraged to ask questions and seek the technical assistance of the teaching assistant -and also receive individual feedback if they chose to upload their completed spreadsheet to the LMS. Teaching assistants did not provide an explanation of the concepts required, as students were expected to have engaged with this intentional content prior to attending the lab. Students were not permitted to watch the videos during lab sessions. At the end of each week, an 'answer' spreadsheet was posted on the module site on the LMS and students could compare their attempt with the solution. Although mandatory to attend labs, no marks for attendance were allocated and this practical component was assessed using only an end-of-term practical exam.

Results and discussion
The findings highlight a discernible pattern of video-based intentional content usage by students. As shown in Table 3, the highest number of students (168) watched 4 out of 5 sets of intentional content videos resources, demonstrating that approximately just over one-third of students had watched all videos right up to the penultimate week. This was followed closely by a further 40 who only watched the videos in week 6. Of significant interest, the results further highlight that approximately one-quarter (106) did not watch any of the videos at any time, bringing to the fore the question of how students were going to complete other aspects of the programme without engagement with this prerequisite material and intentional content? A further 71 are categorised as those who did not watch all the videos, but what they did watch was done in non-consecutive (weeks 7, 8 and 9 but not for weeks 6 and 10 by 10 students), highlighting a non-consistent approach to students engagement. Overall, the pattern of engagement shows a sizeable portion of students not using the video resource consistently from the very beginning with slight increases until week 9 when there was a significant drop in viewing of all the relevant videos. Following the analysis on the usage of the instructional videos, the findings now investigate the level of student engagement with the supporting material that accompanied the instructional videos. This supporting material included instruction handouts and worksheets for completion during the semester.
In Table 4 the accessing of weekly general instructions is compared to student numbers accessing question sheets that described the tasks to be completed in that particular week.
As indicated in Table 4, only a minority of students accessed the general instructions over weeks 6-10. The highest level of engagement emerged for the first week (week 6), albeit a small number of 25. However, the number of students viewing the instructions continued to decrease over the remaining weeks, decreasing to only 5 students viewing the instructions in week 9. Given these results, it was surprising to see that although very low numbers of students had viewed the instructions, an increased number of them actually viewed the question sheet that they were expected to complete. Similar to the instructions, the highest number (131) viewed the question sheet in Week 6; however a week-on-week decrease in its viewing is highlighted with the lowest percentage of students in Week 9 (21.2%). Regarding the viewing of both general instructions and question sheets; while 191 accessed both at the start, this quickly dropped after the first week suggesting both were not seen as necessary. Overall, very few looked at the general instructions without accessing the weekly question sheet, as would be expected. The majority of students began by accessing the question sheet, either with (191) or without (131) general instructions. A large number of students (121) began by looking at neither, growing to 323 by the final week. While the instructions were general, it would not be possible to complete the spreadsheet template without reading the question sheet. Table 5 examines whether students attempted to complete the weekly task (outlined in the question sheet) and the partially filled spreadsheet template. A large, increasing, number of the class (309) did not access either of these files, a consistent pattern over the semester. Similarly, a small number accessed the question sheet but without reference to the spreadsheet template and alternatively some students accessed the spreadsheet template without accessing the question sheet. While in the first week the majority attempted (or at least examined) the weekly task this number dropped consistently and progressively over time.
At the end of each week the students were provided with a solution spreadsheet. This model answer along with the spreadsheet template would allow a student to examine what needed to be done or to check if they had completed the weekly task correctly (Table 6).
While 146 accessed both in the first week this dropped quickly suggesting overall this 'checking' did not occur. While initially most (200) accessed the question sheet but not the solution spreadsheet this quickly changed by the second week with the most students (242) accessing neither file.
Additional cross tabulation analysis was completed to investigate the combined usage of intentional content by students, with a particular focus on video-based learning with the other intentional content i.e. question sheets, spreadsheet templates, and solutions. In Table 7 the use of the question sheet (with weekly tasks) was compared to video viewing categories.
Overall, there was a move from accessing to not accessing the question sheet. For those accessing the question sheet, the majority watched ALL related videos (e.g. even in the last week 106 of 137 accessed all the respective videos). Of those not using the question sheet, the majority (SOME and NONE categories combined) were more than the ALL category.
Furthermore, Table 8 highlights that those students not watching any videos tended to also not access the spreadsheet template. Of those accessing the spreadsheet template (dropping from 346 to 135) they tended to access all the respective videos. There were a  sizeable number of students each week who did not look at the spreadsheet template but watched all the videos. From Table 9, it can be seen that those (small number) who accessed the solution spreadsheet also tended to use all relevant videos. Very few students accessed the solution having watched no videos at all. The number of those not accessing the solution began high (319) and reached 440 out of 468 by the final week. A large proportion of the class watched all the videos without reference to the model answer spreadsheet.

Conclusion & future recommendations
The overall aim of this paper is to investigate the third pillar of the FLIP approach by investigating students' usage and engagement with instructional videos as intentional content and how they serve as a foundation for learning in a FLIP environment. Secondly, the study investigated usage frequencies and specific intervals of engagement, with a view to identifying critical times in the semester where intentional content (i.e. videos and supporting material) are perceived as essential by students. Thirdly, the study explores how intentional content (i.e. instructional videos) supplement other elements of the FLIP environment.
The general findings deducted that students are not proactive in accessing and using intentional content in the form of instructional videos clearly evidenced by their very low level and frequency of engagement with videos and other supporting material (e.g., spreadsheets, spreadsheet solutions). Moreover, it appears students are not consistent in their approach to accessing and viewing instructional videos; instead adopting a rather ad hoc approach and not following instructions as set out by the professional educator. Using instructional videos as the foundation for driving other elements of the FLIP classroom in this particular study assumes a logical and sequential manner where students are encouraged to follow a linear pattern of engagement; in other words, each step completed leads logically to the next i.e., instructional video, instruction handout, worksheet tasks, spreadsheet completion, and model answer. However, this is not the case. What the findings very clearly highlight is that students do not engage in a linear learning process, and do not follow a linear pattern in their learning engagement. Instead, the findings demonstrate that students often completed some deliverables without completing the prerequisites, and alternatively viewed solutions without looking at questions. This brings to the fore the question of the amount of, and level of intentional content viewed as required by students. The findings in some way point to the   fact that students delve straight into the task at hand, not requiring the foundational knowledge provided by the instructional videos and additional supporting instruction handouts. Further findings show that students look straight to solutions without taking the time to go through the process of how these solutions were reached, suggesting in some way that first year students might not yet be prepared for independent and self-regulated learning. Indeed, they may require additional supports and time to gain experience and confidence in developing their learning competency and capabilities.
The intervals at which students generally engage with intentional content (i.e., the instructional videos) appears to be at the beginning and end of the module with consistent disengagement during the intervening period. This proposes that students appear to be engaged at the outset, however motivation levels change, in fact, decreasing as the term progresses. Consequently to this, there is a spike in engagement again at the end of the term, perhaps highlighting that students engage as exam time approaches -really only engaging through necessity rather than being proactive.
This study contributes formal and direct data in an attempt to demonstrate the usage and frequency of usage of intentional content in a FLIP learning environment, at a macro level from log data based on a large sample size of 468 students, advancing the application of learning data analytics in the detection of disengagement with intentional content. Further, the study contributes to the lacuna of empirical quantitative research on the effectiveness of, and students' engagement with instructional videos as a form of intentional content, bringing to the fore important issues for consideration in future studies, given the increase in the delivery of online programmes. Based on the findings of this study, further qualitative studies (e.g., in-depth student interviews; focus groups) are recommended with a view to exploring students' behaviour, attitude and perception with regard to their engagement with intentional content i.e., instructional videos. Such qualitative insights will assist in uncovering richer and more meaningful student opinions on instructional videos as a medium for learning, their level of understanding of material delivered through videos, and the reasons underlying engagement at different intervals in the semester. The triangulation of the log data from this study combined with future qualitative studies will provide a more holistic and powerful insight into of how students engage with intentional content. Future research should also investigate if correlations exist between the engagement with intentional content and exam performance and finally, undertake an analysis of the usage of, and engagement with intentional content other than instructional videos.

Declaration of competing interest
None.