Examining the use of prompts to facilitate self-regulated learning in Massive Open Online Courses

The limited instructional support in Massive Open Online Courses (MOOCs) inherently demands learners to self-regulate their learning. MOOC research shows that learners are more successful when they engage in self-regulated learning (SRL) behaviors such as planning what to study and reviewing study materials. However, many learners struggle with SRL. In this study, we examined the effect of two types of SRL prompts (i.e., questions or a combination of questions and recommendations) on SRL activities, course engagement, and performance in MOOCs. Learners either received questions supporting SRL, questions supporting SRL followed by recommendations, or neither questions supporting SRL nor recommendations. Log data was used to examine learners ’ behavior in the MOOCs. Results showed the SRL prompts, in general, are effective in enhancing SRL-related activities and course engagement. However, the effectiveness of the SRL prompts may be influenced by the complexity of the MOOCs. The current study adds to the field of SRL by examining prompting as an approach to enhance SRL in MOOCs.


Supporting self-regulated learning (SRL) in MOOCs
SRL refers to the pro-active process that learners engage in to optimize their learning outcome (Zimmerman, 2008).According to Zimmerman's model of SRL (Zimmerman & Moylan, 2009), SRL processes can be generally organized as three cyclical phases: forethought, performance, and self-reflection.The forethought phase includes processes related to task analysis, such as goal setting and strategic planning, and self-motivational beliefs.The performance phase includes self-control processes, such as task and attention focusing strategies, and self-observation.The self-reflection phase includes processes involving self-judgment and self-reaction.Schunk and Ertmer (2000) theorized that the quality and quantity of learners' SRL is affected by their employed SRL activities, the frequency in which they engage in the SRL activities, and how well they perform the SRL activities.Research in online contexts indicated that learners who are more self-regulated in their learning tend to achieve greater success in course performance, course satisfaction, and attainment of personal goals (Broadbent & Poon, 2015;Cho & Shen, 2013;Kizilcec, Pérez-Sanagustín, & Maldonado, 2017).However, learners' ability to self-regulate their learning varies and not all learners are highly capable of SRL (Azevedo, 2009;Bjork, Dunlosky, & Kornell, 2013).Therefore, it is important to examine ways to support and enhance SRL to increase learners' likelihood of being successful in MOOCs.
MOOC platforms have some technological features in place to support SRL activities such as deadlines and notifications.Yet, researchers argue that other forms of support, such as prompting learners to selfregulate their learning, are needed to support SRL in MOOCs (Pérez-Álvarez, Maldonado-Mahauad, & Pérez-Sanagustín, 2018;Terras & Ramsay, 2015).There is a small but increasing number of studies on examining SRL supports in MOOCs (Lee, Watson, & Watson, 2019).Kizilcec, Pérez-Sanagustín, and Maldonado (2016) examined the effectiveness of recommending SRL strategies.In the study, half of the learners received a pre-course survey with recommendations of seven SRL strategies (e.g., plan ahead, take notes and summarize the course content to better understand it) while the other half did not receive any recommendations.Results showed that there were no significant differences in the number of videos watched and assessments passed between the learners who received the recommendations and those who did not receive them.As a result, the authors concluded that providing recommendations in a pre-course survey was ineffective and proposed that SRL supports should be integrated more closely with the MOOCs.
Davis, Chen, Van der Zee, Hauff, and Houben (2016) examined the effects of two SRL supports in MOOCs, namely embedded retrieval practice cues in the first experiment and the provision of a study planning activity in the second experiment.Similar to Kizilcec et al.'s (2016) study, there were no significant differences in quiz grades and course engagement between learners who were provided with the SRL supports and those who were in the control condition without any SRL supports.Further analysis showed that learners who actually engaged with the study planning activity achieved higher quiz grades and were more engaged in the MOOC than learners who did not engage with the study planning activity even though they had access to the study planning activity and learners in the control condition.The results suggest that the effectiveness of the study planning activity is influenced by learners' compliance and it is important to consider the frequency and positioning of SRL support with the learning content of the MOOCs.
Considering that content in MOOCs is mainly delivered in the form of videos, Jansen, van Leeuwen, Janssen, Conijn, and Kester (2020) examined an approach to support SRL in the form of videos informing learners about the three phases of SRL according to Zimmerman's SRL model.Three videos were created with information of the SRL phases and recommendations on actions that can be taken for one particular phase per video.Each video was embedded at the end of the learning content and before any quiz in the first three modules of the MOOCs.The study showed that learners who viewed the SRL videos engaged in more SRL activities and had higher course completion than learners who were not provided with the SRL videos.These positive results suggest that informing and suggesting SRL is an effective approach to support SRL in MOOCs.
While the results on the effectiveness of supporting SRL in MOOCs in the above studies are mixed, the results collectively suggest that SRL support in the form of videos or at least a format that is integrated in the MOOCs can be effective.Compared to the extensive research in other online contexts to support SRL, research in MOOCs is only beginning (Wong et al., 2019a)().Based on Devolder et al.'s (2012) review of SRL supports in computer-based learning environments, prompts appear to be an effective approach to enhance SRL.In the next section, we will discuss the effect of prompting SRL in other online contexts to better understand how prompting SRL can be implemented in MOOCs.

Using prompts to promote self-regulated learning
Prompting can be categorised as an indirect instructional method intended to support the recall and use of knowledge and skill, and hence, does not present learners with new information (Bannert & Reimann, 2012).Lehmann, Hähnlein, and Ifenthaler (2014) explained that the goal of prompting is to direct learners' attention to specific aspects of their learning process.As learners think of the effectiveness and efficiency of their current learning strategies, they activate and tap into their repertoire of SRL knowledge and skills.Accordingly, the activated knowledge and skills induce SRL activities during learning.Therefore, prompting to support SRL assumes that learners already have the knowledge and skills but are unable to spontaneously recall or apply their knowledge and skills during learning (Bannert & Reimann, 2012;Berthold, Nückles, & Renkl, 2007).This assumption most likely applies to MOOC learners since the majority of MOOC learners have a higher education degree and would have gained extensive learning experiences through their years of education (Li, 2019).
Prompts serve as strategy activators and can be presented in various ways to achieve the desired effect of helping learners recall and use their SRL knowledge and skill.For example, learners can be prompted with questions to think of their current state of learning (e.g., Do you understand the main points of this week's course materials?)or recommendations of SRL activities that can be employed during learning (e.g., Pace yourself when learning in order to have time to go through all the course materials.).Bannert and Reimann (2012) implemented a pop-up window that appeared at timed intervals (i.e., before, during, and at the end of the learning session) to prompt learners.The prompts included a mix of instructions and questions (e.g., "How do I proceed?Write down how you will check your progress at the end of learning").Results showed that the prompts increased SRL activities in most of the SRL aspects that were prompted.Ifenthaler (2012) compared generic prompts (i.e., stop and reflect) with directed prompts (i.e., instructions to use certain SRL activities) in problem solving tasks with university undergraduates as participants.Results showed the participants in the generic prompt group outperformed those in the directed prompt group and control group.Berthold et al. (2007) found that learners who were prompted with questions (e.g., "Which main points have I already understood well?") had a higher level of understanding and retention of information than learners who were not provided with any prompts.In a more recent study, Müller and Seufert (2018) embedded question prompts that were adapted from Berthold et al.'s (2007) study.Prompted learners outperformed learners who were not prompted, but only in the learning session with the prompts and not in the learning session without the prompts.The results corroborated with Sitzmann and Ely's (2010) study that found that learners who were continuously prompted throughout the learning sessions performed better and were less likely to drop out than learners who were prompted at the initial or last two units.
In general, past research provides evidence that suggests that prompting SRL can be an effective approach to enhance SRL and performance in online learning environments (for a review, see Wong et al., 2019a ).While many studies examining SRL were conducted with university undergraduates, the learning task (e.g., problem solving, Ifenthaler, 2012;Excel training, Sitzmann & Ely, 2010), the type of the prompts (e.g., question, instruction, complete the sentence), and the timing of presenting the prompts varied among the studies.Results from several studies suggest that SRL prompts should be provided throughout learning and across learning sessions to increase uptake and engagement in SRL activities (e.g., Müller & Seufert, 2018;Sitzmann & Ely, 2010).Ifenthaler (2012) also suggests that different types of prompts might be beneficial for different learners, for instance, directed prompts could be more helpful for learners who lack the skills and knowledge needed for learning.Given the open nature of learning in MOOCs, it is not clear whether prompting SRL is an adequate support for MOOCs learners.In addition, MOOC learners are highly diverse (e.g., different age group, educational background, prior knowledge), it is not clear whether providing questions alone is sufficient to foster SRL or a more comprehensive support by supplementing the questions with recommendations would be more effective.

Current study
The current study aimed to examine whether the positive effect of prompting learners in online learning environments would extend to MOOCs.The MOOCs examined were offered on the Coursera platform.On the Coursera platform, learners were allowed to enrol in the MOOCs at any time and they were assigned to a cohort to follow the MOOC on a specific schedule (i.e., fixed start and end dates).The start and end dates were a feature that Coursera put in place to help learners plan their study schedule and to facilitate peer-graded assessments.After learners enrolled in the MOOCs, they had access to all content.Content in the MOOC was divided into modules and the suggested study pace was one module each week.In other words, a MOOC with six modules was also a six-week course.Each week (module), learners were recommended to spend a certain number of study hours.Typical course items in a MOOC were video lectures, texts, discussions, quizzes, and peer review assignments.Depending on the content of the MOOCs, the required amount of study time per week ranged between three to 8 h.Learning in MOOCs was considerably flexible even with the fixed start and end date.Learners were free to progress through the course at their own pace and they only had to pass all graded assessments before the specified end date if they wanted to complete the course in the specified time.Otherwise, they were allowed to reenrol in the MOOC to continue in a new cohort with new start and end dates.Therefore, the learning design described (e.g., access to all course items with little supervision and learners have control of when to learn and how to learn) implied that learning in MOOCs involved learners taking control of their learning and to a large extent by self-regulating their learning (Maldonado-Mahauad, Pérez-Sanagustín, Kizilcec, Morales, & Munoz-Gama, 2018;Weinhardt & Sitzmann, 2019).
The current study consisted of three experiments that were conducted in three different MOOCs (i.e., Serious Gaming, Innovation Management, and Econometrics).The three MOOCs differed not only in disciplines, but also in the type of course activities (e.g., peer review, quizzes, discussions), number of course activities (e.g., the Econometrics MOOC contains almost twice as many course activities than the Serious Gaming MOOC), MOOC duration, and the targeted learners (e.g., the Econometrics MOOC is suitable for advance graduates in the field of economics and finance whereas the Serious Gaming MOOC is suitable for learners who consider a study in digital media).Consequently, the three MOOCs are of a different level of complexity in terms of the topics covered as well as the amount of effort demanded from the learners.By separately examining the three MOOCs across three experiments, we were able to compare whether the effect of prompting could be replicated and would generalize across MOOCs that differed in so many aspects and delve into how learners' in different MOOCs progressed across course weeks.Each experiment is separately presented and discussed in the paper.Two formats of prompting were investigated in this study: prompting SRL with questions or prompting SRL with question followed by recommendations to perform SRL activities.These prompting conditions were compared to a no-prompting control condition.
In the question-prompt (SRL-Q) condition, learners had access to weekly SRL-prompt videos comprising three questions each prompting planning, monitoring, and reflection.Based on the assumption that learners do not spontaneously self-regulate their learning and that past research showed that prompting learners to think about their learning processes in online learning environments positively influenced learners' SRL activities and completion of courses (Bannert & Reimann, 2012;Berthold et al., 2007;Sitzmann & Ely, 2010), it is likely that prompting learners in the form of asking questions in the SRL-Q condition can benefit learners by enhancing their SRL activities, course engagement, and performance.
In the question-prompt and recommendation (SRL-QR) condition, learners had access to weekly SRL-prompt videos comprising three questions as well as three recommendations on SRL activities related to planning, monitoring, and reflection.Based on previous literature, another reason for suboptimal SRL is that learners lack the knowledge of effective SRL activities needed to effectively self-regulate their learning (Bjork et al., 2013;Schunk & Ertmer, 2000).Therefore, supplementing the question-prompts with recommendations on effective SRL activities can be beneficial for learners who lack the knowledge to effectively self-regulate their learning and need recommendations on the SRL activities that can potentially enhance their learning in the MOOC.Finally, in the control condition, learners had no access to any of the SRL-prompt videos in the SRL-Q and SRL-QR condition.
The first research question concerned SRL behavior in MOOCs and was formulated as "Does prompting SRL (in the form of questions or in the form of combining questions with recommendations) enhance SRL-related activities in MOOCs?".The prompts and recommendations implemented in the study were intended to support SRL activities (i.e., planning, monitoring, and reflection) according to the three phases of SRL (Zimmerman & Moylan, 2009).Therefore, we identified four types of behavior from the log data (i.e., access to course preparatory materials, number of visits of grade information page, proportion of course items completed on time, and proportion of completed course activities that were repeated) as proxies of SRL-related activities in accordance to the three phases of SRL.In each SRL-prompt video, the first prompt and recommendation targeted planning (e.g., Set clear learning goals on what you want to learn and make plans to achieve them).Prompts and recommendations along these lines would require learners to gather information about the course in order to make plans on what to study and how to study.In MOOCs, such information can be obtained from the course preparatory materials provided, specifically the introductory readings or videos, the course overview page, and weekly course information pages.According to You (2016) reading course information packets significantly predicted course achievement, and in Jansen et al.'s (2020) study learners who watched the SRL intervention videos visited overall and weekly course information pages more often than learners in the control condition.Therefore, we identified accessing course preparatory materials provided in the MOOC as indicative of planning in the forethought phase of SRL.The second prompt and recommendation targeted monitoring of learning (e.g., Am I concentrating on learning the materials in this course?)and time (e.g., Am I trying to schedule time to study for this course and observe the schedule as much as possible?).Such prompts and recommendations direct learners to think about how well they are progressing and whether they are on schedule.One of the ways to monitor one's progress in a MOOC is to access the grade information page to check how well one had scored in the graded assessments and what other assessments one had to pass to complete the MOOC (Jansen et al., 2020).Therefore, we used learners' number of visits to the grade information page as indicative of a form of self-monitoring of learning.While learning in MOOCs is considerably flexible, learners were given suggested deadlines according to the recommended study pace (i.e., one module per week).Keeping up with the pace of the course and duly completing each week's course activities may imply that learners are able to manage their time well (Jansen et al., 2020;You, 2016).Therefore, as an indicator of time-management (i.e., monitoring of time), we used the proportion of course activities in the MOOC that was completed on time.Finally, the SRL videos included prompts and recommendations that targeted reflection (e.g., Have I spent enough time reviewing the videos and doing the activities to remember the information in this course?).Such reminders might prompt learners to revisit course materials that they have previously completed to strengthen their understanding (Kizilcec et al., 2017).Therefore, we used the proportion of completed course activities that were repeated as an indicator of self-reflection.
Learners in the SRL-QR condition received both prompts and recommendations, and hence, not only were they stimulated to act in a self-regulated manner by the prompts (Sitzmann & Ely, 2010), the recommendations also helped to inform them of possible SRL activities that can be done to enhance their learning (Jansen et al., 2020).Given this additional guidance in the SRL-QR condition, we hypothesized that learners in the SRL-QR condition would engage in the most number of SRL-related activities as measured by the log data (i.e., planning as indicated by the highest number of access to course preparatory materials, self-monitoring as indicated by the highest number of visits to the grade information page, time-management as indicated by the highest proportion of course items completed on time, and self-reflection as indicated by the highest proportion of completed course items that were repeated), followed by learners in the SRL-Q condition, and then learners in the control condition (Hypothesis 1A to 1D).
The second research question concerned learner engagement in MOOCs and was formulated as "Does prompting SRL (in the form of questions or in the form of combining questions with recommendations) enhance course engagement in MOOCs?".We defined learner engagement as the proportion of course items available in the course that were completed and the average number of course items that were accessed for each active day that the learner was in the course.We hypothesized that learners in the SRL-QR condition would complete the highest proportion of course items available in the course and access the most number of course activities for each active day in the course, followed by learners in the SRL-Q condition, and then learners in the control condition (Hypothesis 2A and 2B).
The third research question concerned course performance in the MOOCs and was formulated as "Does prompting SRL (in the form of questions or in the form of combining questions with recommendations) enhance course performance in MOOCs?".We defined course performance as the overall course grade that learners received from all the graded assessments (i.e., quizzes and peer-review assignments) in the course.We hypothesized that learners in the SRL-QR condition would have the highest course grade, followed by learners in the SRL-Q condition, and then learners in the control condition (Hypothesis 3).
The current study contributes to the research in the field of SRL and MOOCs in several ways.Firstly, the study is one of the few studies at present to empirically examine interventions in MOOCs.Secondly, the study examined the effects of prompts in three different MOOCs to better understand whether the effect of SRL prompts can be generalized across different MOOCs.Thirdly, the study examined two types of prompts (i.e., questions only and a combination of questions and recommendations) to understand how different types of prompts influence learning in MOOCs.Finally, the study utilized log data to not only examine learners' progress in the course but also to explore learners' sequences of learning behavior in relation to the provision of SRL prompts in the MOOCs.

Experiment 1: Serious Gaming
The first experiment was conducted in a six-week MOOC on the topic of Serious Gaming (SG).The SG MOOC was designed as an introduction to the concept, application, and impact of serious games.There were six modules in the course, one for each week.The recommended study time for each module was three to 5 h.Besides the course preparatory materials (i.e., course overview page and weekly content page, course introductory video, reading about the team), there were 46 course items that learners could access in the course to learn about serious gaming.The 46 course items included 24 videos, 6 discussions, 8 texts, 1 ungraded quiz, 1 ungraded peer-review, 4 graded quizzes, and 2 graded peer-review assignments.

Participants
Data were collected in three consecutive cohorts of the SG MOOC.Coursera collects and stores data for analytics described in their terms of use and privacy policy.The collected data were then made available for research purposes via the signed partnership agreement between the university that offered the MOOCs and Coursera.A total of 501 enrolled learners were randomly assigned to one of the three conditions.Many MOOC learners do not start their course after enrolment (Davis et al., 2016).That is, the number of learners who eventually click on an activity in the course (i.e., active learners) is much lower than the number of learners who enrolled (i.e., enrolees).Furthermore, learners are free to do as many or as little course activities as they want.Therefore, there is also a difference between the number of learners who have access to the SRL-prompt videos (i.e., learners in the SRL-Q and SRL-QR conditions) and the number of learners who watched at least one SRL-prompt videos (i.e., SRL-prompt viewers).In other words, SRL-prompt viewers are a subset of the active learners in the two types of SRL-prompt conditions.
Table 1 shows the distribution of learners across the three conditions and the categorization of the learners.In the control condition, 44.2% of them were identified as active learners who accessed at least one course item.In the SRL-Q and SRL-QR conditions, 55% and 52.7% of the enrolees in the respective conditions were identified as active learners.
To understand the effect of the SRL-prompt videos, we identified active learners in the SRL-Q and SRL-QR conditions who watched at least one SRL-prompt video (i.e., SRL-prompt viewers).Of the active learners, 26.6% and 40.9% viewed at least one of the SRL-prompt videos in the SRL-Q and SRL-QR condition respectively.

Materials and procedure
Learners were randomly assigned to one of the three conditions (i.e., SRL-Q, SRL-QR, and control) by the Coursera platform when they enrolled in the SG MOOC.At the point of enrolment, learners in all three conditions received a general message that informed them that the course version they were enrolled in would be used to investigate learning in MOOCs and there would be some materials to support their learning in the course.
After enrolling, learners in all three conditions would proceed with taking the MOOC as usual with access to all the course materials that were available in the course itself.Learners also had access to a presurvey that was placed in Module 2 and a post-survey that was placed in Module 6.The pre-and post-surveys measured learners' motivation and SRL for both SRL-Q and SRL-QR conditions.Learners in the control condition had access to the same post-survey.However, the pre-survey for learners in the control condition included only items measuring motivation.The main difference between the control and SRL-prompt conditions (i.e., SRL-Q and SRL-QR conditions) were the presence of the self-regulated learning (SRL) prompt videos.

SRL-prompt videos.
Two different SRL-prompt videos were created for the SRL-Q and SRL-QR conditions to prompt SRL.We took three main factors into consideration when designing the SRL-prompt videos.The first factor was what to prompt.According to Zheng's (2016) meta-analysis, supporting the whole SRL process was more beneficial than supporting one specific phase of SRL (d = 0.469).Therefore, each video contained three questions that were intended to activate planning, monitoring, and reflection according to Zimmermann's (2000) SRL model.The second factor was when to prompt.Sitzmann and Ely (2010) found that continuous prompting throughout a free online training program was more effective than prompting in the first few weeks or last few weeks of the course.Moreover, MOOC learners are free to start on any course module.Therefore, we created SRL-prompt videos for each module of the course to prompt learners throughout the course.The third factor was where to place the SRL-prompt videos.In order to reach learners in the middle of their learning session, we placed the SRL-prompt videos after the second video lecture in each module.
All SRL-prompt videos began with the same message below which was adapted from Sitzmann and Ely's (2010) study to explain the importance of thinking about one's learning process: Research shows that asking yourself questions about how well you are planning, monitoring, and reflecting on your learning will increase how much you learn during the course.Several times throughout this course, you will be asked three questions about how well you are learning.Honestly respond to these questions by selecting the option that best reflects your learning state.Use your responses to improve your learning during the course.
After the introductory message, the SRL-prompt videos in the SRL-Q condition sequentially presented three questions to prompt SRL with pauses in between for learners to respond to the questions on a 5-point scale from 1 (not at all) to 5 (all the time).The questions and recommendations used in the SRL-prompt videos are included in Appendix 1. Fig. 1 shows a series of screenshots illustrating the main frames from one of the SRL-prompt videos in the SRL-QR condition.
4. 1.3. Measures 4.1.3.1. Motivation survey.Learners had access to a pre-motivation survey in Week 2 and a post-motivation survey in the last week of the MOOC.An email announcement was sent during those weeks to encourage learners to complete the survey.The motivation survey was adapted from Ryan and Connell's (1989) study and has been used in previous research (e.g., Vansteenkiste, Sierens, Soenens, Luyckx, & Lens, 2009).The 16-item scale is made up of four subscales with four items each measuring intrinsic, identified, introjected, and extrinsic regulation of motivation.Learners indicated on a five-point Likert scale ranging from 1 (completely not important) to 5 (very important) the personal importance of each of the 16 statements.According to Self-Determination Theory (Ryan & Deci, 2000), autonomous motivation consisted of intrinsic motivation and well-internalized extrinsic motivation (i.e., identified motivation) while controlled motivation consisted of two externally regulated forms of motivation (i.e., introjected and extrinsic motivation).Therefore, we averaged the scores from the subscales of intrinsic and identified regulation of motivation to form a composite score for autonomous motivation and the subscales of introjected and extrinsic regulation of motivation to form a composite score for controlled motivation.The same method of forming composite scores were used in several prior studies (e.g., Vansteenkiste et al., 2009; Cronbach's alphas for autonomous motivation was 0.87 and for controlled motivation was 0.72).The subscales of autonomous and controlled motivation in our study also had high reliabilities, Cronbach's α = 0.79 and 0.76 respectively.

SRL survey.
Besides the motivation survey, learners were provided with a pre-and post-SRL survey except in control condition's Week 2 survey.The SRL survey consisted of 29 items across seven scales: goal setting (4 items; α = 0.75), strategic planning (4 items; α = 0.66), task strategies (6 items; α = 0.61), and self-evaluation (3 items; α = 0.58), which were taken from Littlejohn, Hood, Milligan, and Mustain's (2016) study, time-management (3 items; α = 0.51) and environment structuring (4 items; α = 0.80), which were taken from Barnard-Brak et al.'s (2010) study, and persistence (5 items; α = 0.81) from Jansen, Van Leeuwen, Janssen, Kester, and Kalz's (2017) study.Learners were asked to indicate on a 5-point Likert scale ranging from 1 (not at all true for me) to 5 (very true for me) how typical the learning behaviors associated with each scale was for them.While the motivation survey measured the degree to which learners motivation was autonomously or controlled, the SRL survey measured learners' perceived level of SRL (e. g., to what extent do they self-regulate their learning).

SRL-related activities.
We identified four proxies of SRL as indicated by learners' behavior measured from the log data.As mentioned in the section describing the current study, we operationalized the first SRL-related activity as an indicator of planning using the number of course preparatory items accessed (i.e., sum of course introductory videos watched and the number of visits to the course overview and weekly course information pages).The second SRL-related Fig. 1.Screenshots of the main frames from one of the SRL-prompt videos in the SRL-QR condition.
activity was self-monitoring and was operationalized by the number of visits to the grade information page.The third SRL-related activity was time-management and we operationalized time-management by the proportion of course items that were completed on time (i.e., sum of course items completed on time divided by the total number of course items in the course).The fourth SRL-related activity was self-reflection and we operationalized self-reflection by the proportion of completed course items that were repeated (i.e., sum of completed course items in the course that were repeated divided by the total number of course items in the course).

Course engagement.
MOOCs have a number of course items that learners can access for their learning (e.g., video, quizzes).In our study, course engagement is operationalized by two measured outcomes: 1) the proportion of course items accessed by learners and 2) the average number of course items accessed by learners per active day in the course.The proportion of course items accessed by the learners is calculated by the sum of unique course items accessed by a learner divided by the total number of course items in the course, indicating the extent to which the learners have covered the content of the course.On the other hand, the average number of course items accessed by learners per active day in the course is calculated by a learner's total frequency of access to course items including repetitions divided by the number of days the learner was active in the course.The measured outcome gives an indication of a learners' average activity level in the course.Therefore, the two measured outcomes provide a different perspective on course engagement.

Course performance.
The final course grade (calculated by the sum of the graded assessments multiplied by the weights assigned to each graded assessment) was provided by Coursera.Learner's final course grade is indicative of how well learners have performed in the course.

Data processing and analysis
The survey data were collected via Qualtrics (https://www.qualtrics.com) while the learner behavioral data were retrieved from the Coursera's database.A total of 109 completed responses for the pre-survey and 33 completed responses for the post-survey were collected across all three conditions.Consequently, we analyzed only the level of motivation in the pre-survey as a randomization check to ensure that the learners in three conditions did not differ in their level of motivation at the beginning of the MOOC.The survey data violated the assumption of normality, and hence, we employed non-parametric Kruskal Wallis test separately for autonomous and controlled motivation.Results show that learners in three conditions at the beginning of the MOOC did not differ significantly in autonomous motivation, H (2) = 0.07, p = .97,nor controlled motivation, H (2) = 2.09, p = .35.For completeness, we reported the means and standard deviations of the motivation and SRL subscales measured at Week 2 and final week of SG MOOC in of Appendix 2.
For the learner behavioral data, we exported data tables from Coursera that included tables on the sessions of the courses, course content documenting the course items offered to the learners, course progress documenting the course items started and completed by the learners, course grades documenting learners' grades for each assessment, and access data documenting the course pages viewed from the web browser by the learners.All downloaded data were imported into R and anonymised using the crsra package developed by Hadavand and Leek (2018).
The first phase of data processing consisted of four steps.We first identified learners enrolled in the three cohorts of the MOOC during the period in which the experiment was being conducted using a unique identifier assigned to each cohort of the MOOC.Learner IDs that appeared in multiple cohorts of each MOOC (i.e., learners who reenrol) were removed from the analysis as these learners could have been exposed to multiple experimental conditions.The second step involved filtering learners' actions with all course items from the course progress data table (e.g., started a video, completed a video) and learners' access to the course pages from the access data.Given that learners could enrol in a MOOC at any time and continue to access the course items even after the course ends, we used the start date of course enrolment till the end date of the course for each cohort as the cut-off dates to allow for a fair comparison of learners' activity in the course within the same duration.In the third step, we calculated the dependent variables based on the operationalization of the learners' behaviors described in the section above (e.g., learner engagement defined by the proportion of course items accessed by learners).In the final step, we created a data table with all the dependent variables and the conditions that the learners were assigned to.The Rscripts used for processing the data are available upon request.
The data were prepared for two types of analyses in the second phase of data processing: intent-to-treat (ITT) and treatment-on-treated (TOT) analyses (Lamb, Smilack, Ho, & Reich, 2015).For the ITT analysis, we identified learners who have accessed at least one course item (i.e., active learners) excluding the course preparatory items (i.e., introductory videos and readings).The ITT analysis allowed us to compare active learners across the three conditions regardless of whether the learners in the SRL-prompt conditions (i.e., questions, SRL-Q; questions and recommendations, SRL-QR) watched any of the SRL-prompt videos.For the TOT analysis, we identified learners in the SRL-Q and SRL-QR conditions who completed at least one of the SRL-prompt videos.The TOT analysis allowed us to compare the active learners in the control condition who accessed at least one course content item and active learners in the SRL-Q and SRL-QR conditions who not only accessed at least one course content item but also completed at least one SRL-prompt video (i.e., SRL-prompt viewers).
In view of deviations from normality in the data and unequal sample sizes across conditions, we used robust one-way analysis of variance (ANOVA) based on 20% trimmed mean to compare each dependent variable across the three conditions (Mair & Wilcox, 2016).However, when robust ANOVA with 20% trimmed mean cannot be applied because of Winsorized variance of 0, we reported results from the non-parametric Kruskal-Wallis test in the main analysis.We used the explanatory measure of effect size, ξ, as an alternative to Cohen's d.The ξ values of 0.15, 0.35, and 0.50 correspond to small, medium, and large effect sizes respectively (Wilcox and Tian, 2011).The results of the Kruskal Wallis test for each of the outcome measures across the three experiments are included as a supplementary analysis in Appendix 3. Pairwise comparisons were carried out using Dunn's test with Bonferroni correction for adjusted p-values to further examine the differences between the conditions.We also included an exploratory analysis of sequences of learners' behavior across weeks by employing process mining with the DISCO software (https://fluxicon.com/disco/).

Results
Table 2 shows the distribution of the number of learners across the number of SRL-prompt videos watched in the SG MOOC.The number of learners who viewed at least one SRL-prompt video in the SRL-Q and SRL-QR condition is comparable to the compliance rate of learners in other intervention studies in MOOCs (Davis et al., 2016;Jansen et al., 2020).Table 3 shows the means, standard deviations and trimmed means at 20% for each of the dependent variables across the conditions that were compared.

SRL-related activity
For the ITT analysis, results showed that there was no significant effect of condition on access to course preparatory items, F t (2, 90.63) = 1.36, p = .26,ξ = 0.12, and proportion of completed course items repeated, F t (2, 89.52) = 0.75, p = .48,ξ = 0.17.However, there was a significant effect of condition on the number of course items that were completed on time, F t (2, 98.79) = 3.66, p = .03,ξ = 0.20.Post hoc tests revealed a significant difference between the control and SRL-Q conditions (95% CI [0.002, 0.06], p = .03),but not between the control and SRL-QR conditions (95% CI [-0.02, 0.04], p = .44)and also not between the SRL-Q and SRL-QR conditions, 95% CI [-0.05, 0.01], p = .18).This suggests that active learners in the control condition completed more course items on time than active learners in the SRL-Q condition.We were unable to apply robust ANOVA with 20% because of Winsorized variance of 0 and because only a small number of the active learners in the three conditions visited the grade information page (control, n = 11; SRL-Q, n = 7; SRL-QR, n = 4).We employed the Kruskal-Wallis test and results revealed no significant differences in the grade information page views among the three conditions, H (2) = 5.86, p = .05.
Similarly, there was a significant difference in proportion of course items completed on time across conditions, F t (2, 26.71) = 10.27,p < .001,ξ = 0.40.Follow-up post hoc tests for the proportion of course items completed on time revealed a significant difference between the control and SRL-Q conditions, (95% CI [-0.24, − 0.03], p = .007),and between the control and SRL-QR conditions (95% CI [-0.11, − 0.02].p = .007),but not between the SRL-Q and SRL-QR conditions, (95% CI [-0.04, 0.18], p = .11).There were no significant effects of condition on the number of visits to grade information page, H (2) = 2.34, p = .31,and the number of completed course items that were repeated, F t (2, 29.39) = 2.16, p = .13,ξ = 0.36.The results suggest that SRL-prompt viewers in the SRL-QR condition accessed more course preparatory items.Also, SRL-prompt viewers in both SRL-Q and SRL-QR conditions had higher number of course items completed on time than active learners in the control condition.

Course engagement
Results from the ITT analysis showed that there was no significant effect of condition on either the proportion of course items accessed in the MOOC, F t (2, 87.36) = 0.75, p = .47,ξ = 0.12, or the average number of access to course items per active day, F t (2, 92.5) = 1.55, p = .22,ξ = 0.19.In contrast, results from the TOT analysis revealed significant differences for the proportion of course items accessed in the MOOC, F t (2, 34.18) = 5.54, p = .008,ξ =. 40, as well as the average number of access to course items per active day across the three conditions, F t (2, 38.72) = 4.16, p = .02,ξ =. 34.Post hoc tests for the proportion of course items accessed revealed significant differences between the control and SRL-Q conditions (95% CI [-0.36, − 0.03], p = .02),but not between the control and SRL-QR conditions (95% CI [-0.18, 0.003], p = .04),nor between the SRL-Q and SRL-QR conditions (95% CI [-0.05, 0.27], p = .10).Post hoc tests for the average number access to course items per active day revealed significant differences between the control and SRL-Q conditions (95% CI [-6.97, − 0.49], p = .02),but not between the control and SRL-QR conditions (95% CI [-3.81, 1.67], p = .34),nor between the SRL-Q and SRL-QR conditions (95% CI [-0.58, 5.90], p = .09).The results suggest that SRL-prompt viewers in the SRL-Q condition but not in the SRL-QR condition had higher course engagement than active learners in the control condition.

Course performance
The results on course performance should be interpreted with caution given that very few active learners in the three conditions obtained a course grade (control, n = 13, SRL-Q, n = 15, SRL-QR, n = 9).In the ITT analysis, results from the Kruskal-Wallis test showed that there was no significant effect of condition on course grade, H (2) = 2.48, p = .29.However, a significant difference was obtained in the TOT analysis, H (2) = 12.11, p = .002.Dunn's pairwise comparisons test with Bonferroni correction showed that SRL-prompt viewers in the SRL-Q condition obtained significantly higher course grades than active learners in the control condition (p = .002)and SRL-prompt viewers in the SRL-QR condition (p = .02).No significant differences in course grades were found between the SRL-QR and control (p = 1.0).

Process mining
Using the Disco software, we created process maps to explore sequences of course items that were completed on time by the learners.Fig. 2 illustrates the overview of the interactions with the course items by all learners in the Disco software.After removing interactions with course items that were not completed on time, we identified 195 learners with a total of 2338 (22%) interactions with the course items that were completed on time.The number of interactions included revisiting of course items that were already completed.In the process mining analysis, we divided the learners in the SRL-Q into two groups: learners who did not watch any SRL prompt videos (i.e., non-viewers) and those who watched at least one SRL prompt video (i.e., viewers).The same was applied to learners in the SRL-QR condition, resulting in

Table 3
Means, standard deviations and trimmed (tr.) means for the conditions compared in the ITT analysis (based on active learners in all three conditions) and TOT analysis (comparing SRL-Prompt viewers in SRL-Q and SRL-QR conditions and active learners in the control condition) in SG MOOC.ITT  five groups of learners: control, SRL-Q prompt non-viewer, SRL-Q prompt viewer, SRL-QR prompt non-viewer, and SRL-QR prompt viewer.Table 4 shows the distribution of learners and frequency of interactions with course items that were completed on time in the five identified groups.Fig. 3a-e are process maps illustrating the sequences of interactions with the completed course items on time over six weeks of the SG MOOC.The first observation is the differences in the frequency of access to complete course items on time by the number of learners in the identified groups.While the number of SRL-Q prompt viewers was the smallest, the average access to complete course items on time was the highest.The second observation is related to the distribution of the frequency of access to complete course items on time across the six weeks in the MOOC.The darker the blue in the process maps, the higher the frequency of access to the course items completed on time.It is not surprising that in all five process maps, Week 1 is in dark blue, suggesting that learners typically access Week 1's course items and complete them on time the most.However, only the process map of SRL-Q prompt viewers (Fig. 3c) is in dark blue for Week 2. Furthermore, the process map of SRL-Q prompt viewers shows that the frequency of access to the course items completed on time is still relatively high in Week 3 and also in Week 6.The third observation relates to the number of the learners who started and ended the learning process across the weeks in the MOOC.The green dotted lines represent the start of the process and the red dotted lines represent the end of the process.The process maps show that more than 80% of the SRL-Q and SRL-QR prompt non-viewers (Fig. 3b and d) started the process in Week 1 and ended the process in Week 1, followed by 68.85% of the learners in control condition, then 46.88% of the SRL-QR prompt viewers (3e), and 28.57% of the SRL-Q prompt viewers (Fig. 3c).The fourth and final observation relates to the sequences of access to complete course items on time across the weeks.The five process maps suggest that learners typically follow course weeks in sequence.However, it is observed that there are more instances in which learners skipped the course weeks (e.g., complete course items in Week 1 to complete course items in Week 6) in control condition (Fig. 3a).Similarly, the process maps of SRL-Q and SRL-QR prompt non-viewers (Fig. 3b and d) also show that there is one instance in which completing course items in Week 1 was followed by completing course items towards the end of the MOOC, skipping Weeks 2, 3, and 4. The process map of SRL-Q prompt viewers (Fig. 3c) indicates a more linear sequence of completing course items across the six weeks and less of skipping course weeks or returning to previous weeks.

Discussion
In the first experiment, we examined the effect of prompting SRL in a six-week MOOC.We failed to confirm any of the Hypothesis 1A to 1D based on the ITT analysis as there were no significant differences in any of the four SRL-related activities across the three conditions.An unexpected finding in the ITT analysis is that active learners in the control condition appeared to have completed a greater proportion of course items on time than active learners in the SRL-Q condition regardless of whether an SRL-prompt video had been viewed.However, this finding was contradicted by the TOT analysis where support was found for Hypothesis 1C: SRL-prompt viewers in the SRL-Q and SRL-QR conditions completed more course items on time indicative of better time management than active learners in control condition.Results from the TOT analyses also indicate that SRL-QR prompt viewers access the course preparatory items more than active learners in the control condition.Results from Experiment 1 are aligned with Jansen et al.'s (2020) study that showed that supporting SRL in MOOCs enhances learners' planning   2020) study where learners who complied with the SRL support did not complete more course activities on time, we found that learners who viewed the SRL-prompt videos regardless of the type of prompts (i.e., questions only or questions followed by recommendations) completed more course activities on time, suggesting that both types of prompts supported time management.
For the second research question on course engagement, the results showed that only SRL-Q prompt viewers, but not SRL-QR prompt viewers, completed a greater proportion of the course activities and accessed more course items on average for each active day in the course than active learners in the control condition.The results suggest that SRL support in the form of question-prompts is effective in enhancing course engagement in MOOCs.One possible explanation could be that providing question-prompts is sufficient to elicit SRL.According to Sitzmann and Ely (2010), the questions can help to override tendencies of off-task thoughts and direct one's attention to current learning activities.However, it is not clear why the effect of the question-prompts when coupled with recommendations were not as effective as question-prompts alone.
For course performance (Research Question 3), we found a significant difference in the TOT analysis supporting Hypothesis 3: SRL-Q prompt viewers obtained a higher course grade than active learners in the control condition.However, the results should be interpreted with caution due to the low number of learners who obtained a non-zero course grade in the SG MOOC.
The exploratory process mining was used to further examine learners' process of completing course items on time in the six-week MOOC.We observed that SRL prompt viewers had a high sustained level of access to complete course items on time over the weeks in the MOOC compared to the other groups and the percentage of SRL-Q prompt viewers who ended the process in Week 1 is the lowest.In addition, the process map of SRL-Q prompt viewers (Fig. 3c) indicated that learners' access to complete course items on time follow the sequences of the course weeks with little skipping of course weeks in between (Wong et al., 2019b).The observed differences in the process maps substantiate the findings from the log frequency analysis, suggesting that SRL-Q prompt viewers are involved in more SRL activities (e.g., time planning, persistence) and are more engaged in the course.However, the number of learners who watched at least one SRL-prompt video in the SG MOOC was relatively low.Therefore, the effect of SRL-prompt videos should be further investigated in other MOOCs.

Experiment 2: Innovation Management
The second experiment was conducted in a nine-week MOOC on the   items in the IM MOOC were 31 videos, 5 discussions, 3 ungraded quizzes, and 1 optional reading list.To pass the MOOC, learners had to pass 3 graded peer-review assessments and 1 final graded quiz.Altogether, there were 44 course items distributed across nine weeks.Just like in the SG MOOC, the main type of course item for learning is videos.In comparison to the SG MOOC, the IM MOOC had more videos and less readings.Also, while the SG MOOC had graded assessments almost in every module, the graded assessments in the IM MOOC were placed in alternate weeks in Modules 3, 5, 8, and 9.

Participants
Data was collected over three cohorts of learners enrolled in the IM MOOC.A total of 1319 enrolled learners were randomly assigned to one of the three conditions.The distribution of learners across the three conditions (i.e., control, SRL-Q, SRL-QR) and learner categories (i.e., enrolees, active learners, and SRL-prompt viewers) is illustrated in Table 5. Enrolees were considered as active learners only when they went on to access at least one course activity.In the control condition, 52.9% of the enrolees were active learners.Similarly, in the SRL-Q condition, 52.4% were active learners and in the SRL-QR condition 55.1% were active learners.Of the active learners, 63.6% in the SRL-Q condition and 60.1% in the SRL-QR condition viewed at least one SRLprompt video (i.e., SRL-prompt viewers).

Materials and procedure
The procedure was kept the same as in Experiment 1. Learners enrolled in the IM MOOC were randomly assigned to one of the three conditions (i.e., SRL-Q, SRL-QR, and control) by the Coursera platform and received the same generic message informing them that the version that they were enrolled in would be used to investigate their learning in MOOCs.After enrolment, learners proceeded with the course as they normally would and had access to all course materials.One SRL-prompt video was embedded in each of the eight learning modules in the MOOC.Just like in Experiment 1, the SRL-prompt videos in the SRL-Q condition consisted of three questions each while the SRL-prompt videos in the SRL-QR condition consisted of three questions followed by three SRL recommendations each.

Measures and data analysis
The measures for Experiment 2 were the same as for Experiment 1.We employed the same data processing and analytical procedure as described in Experiment 1.In Experiment 2, we collected 255 completed responses in the motivation and SRL survey in Week 2 and 89 completed responses in the motivation and SRL survey in the final week of the MOOC.Following the same procedure in Experiment 1, we analyzed only the autonomous and controlled motivation subscales in Week 2 to check that the three conditions did not differ significantly in their motivation at the beginning of the course.Kruskal-Wallis test was employed due to the violation of normality.Results show that learners in three conditions at the beginning of the MOOC did not differ significantly in autonomous motivation, H (2) = 0.85, p = .65,and controlled motivation, H (2) = 3.98, p = .14.The means and standard deviations of the motivation and SRL subscales measured in Week 2 and the final week of the IM MOOC are reported in of Appendix 2. All analyses began with the intention to treat (ITT) analysis followed by the treatment on treated (TOT) analysis.

Results
The distribution of number of learners across the number of SRLprompt videos viewed in the IM MOOC is illustrated in Table 6.Means, standard deviations, and trimmed means at 20% for each of the dependent variables across the conditions that were compared are presented in Table 7.
For the TOT analysis, no significant effect of condition was found on access to course preparatory items, F t (2, 162.89) = 1.25, p = .29,ξ = 0.09, and visits to grade information page, F t (2, 175.67) = 2.10, p = .13,ξ = 0.10.However, there was a significant effect of condition on proportion of course items completed on time, F t (2, 177.31) = 3.60, p = .03,ξ = 0.15, and proportion of completed course activities that were repeated, F t (2, 157.76) = 5.27, p = .01,ξ = 0.15.Post hoc tests for the proportion of course items completed on time showed that there was a significant difference between the control and SRL-QR conditions (95% CI [-0.19, − 0.01], p = .04)but not between the control and SRL-Q conditions (95% CI [-0.14, 0.03], p = .20)and also not between the SRL-Q and SRL-QR conditions (95% CI [-0.14, 0.06], p = .37).Similarly, for the proportion of completed course activities that were repeated, post hoc tests showed that there was a significant difference between the control and SRL-QR conditions (95% CI [-0.07, − 0.005], p = .02)but not between the control and SRL-Q conditions (95% CI [-0.07, 0.001], p = .04)and also not between the SRL-Q and SRL-QR conditions (95% CI [-0.05, 0.04], p = .83).The results suggest that learners in the SRL-QR condition, but not the SRL-Q condition, completed a greater proportion of course items on time and revisited a greater proportion of completed course activities than active learners in the control condition.

Course engagement
Results from the ITT analysis showed that there was no significant effect of condition on both proportion of course items accessed in the MOOC, F t (2, 277.23) = 0.76, p = .47,ξ = 0.06, and the average number of access to course items per active day, F t (2, 280.87) = 0.46, p = .63,ξ = 0.05.For the TOT analysis, while a small significant effect of condition was found on proportion of course items accessed, F t (2, 173.73) = 3.38, p = .04,ξ = 0.14, no significant effect of condition was found for the average number of access to course items per active day, F t (2, 190.96) = 2.39, p = .09,ξ = 0.12.Post hoc tests for the proportion of course items accessed did not reveal any significant differences between the control and SRL-Q conditions (95% CI [-0.24, 0.02], p = .08),between the control and SRL-QR conditions (95% CI [-0.24, 0.01], p = .08),and between the SRL-Q and SRL-QR conditions (95% CI [-0.15, 0.15], p = .96).

Course performance
Results on course performance should be treated with caution since there were 54, 41, and 43 learners who obtained a non-zero course grade in the control, SRL-Q, SRL-QR conditions respectively.For the ITT analysis, we were unable to apply a robust ANOVA with 20% trimmed means on the data.Results from the Kruskal-Wallis test revealed a nonsignificant difference in course grade obtained by learners across the three conditions, H (2) = 1.21, p = .54.For the TOT analysis, results from robust ANOVA with 20% trimmed means also showed that there was no significant effect of condition on course grade, F t (2, 118.72) = 1.02, p = .36,ξ = 0.07.

Process mining
Following the steps in Experiment 1, we first generated process maps using all the active learners access to the course items.Of the 705 active learners, 558 learners completed at least one course item on time.Of the 58500 access to course items, 14727 of these are course items completed on time (25.23%).Table 8 shows the distribution of the learners and access to complete course items on time by the five identified groups.Fig. 4a-e are process maps illustrating the sequences of access to complete course items on time over the nine week IM MOOC.The full process maps were challenging to examine due to the "spaghetti-like" processes, suggesting large variation and unstructured behavior in the MOOC (van der Aalst & Gunther, 2007, pp. 3-12).We set the paths in the process maps at 50% to reveal half of the most dominant paths to examine the access of course items that were completed on time by the five identified groups of learners.
The first observation is that the average frequency of access to complete course items on time is the highest for SRL-QR prompt viewers, whereas the average frequency of access to complete course items on time by learners in the control condition is comparable to SRL-Q prompt viewers (See Table 8).SRL-Q and SRL-QR prompt non-viewers had the lowest frequency of access to complete course items on time.Next, the differences in the frequency of access to complete course items on time by course weeks across the five process maps are considered.A similar pattern is observed for viewers in the control and SRL-QR conditions (Fig. 4a and e) where the access to complete course items on time remained relatively high for the first five weeks of the course.For the ninth week of the course in which a course project is due, the process maps for SRL-Q and SRL-QR prompt viewers (Fig. 4c and e) show a bright blue color compared to the control condition, indicating that the access to complete the course project on time is higher when compared to the other groups of learners.Third, we looked at the number of learners who started and ended the learning process in Week 1, which is represented by the green and red dotted lines respectively.Around 80% of the learners who started Week 1 also ended in Week 1 for SRL-Q and SRL-QR prompt non-viewers (Fig. 4b and d).The proportion of learners who started the learning process in Week 1 and ended in Week 1 is much lower in the other three groups (Control, 44.94%; SRL-Q viewers, 44.54%, and SRL-QR viewers, 38.89%).Finally, we examined the sequences of completing the course items on time across weeks.We observed that in the IM MOOC, there were a number of instances in which learners skipped forward in the course weeks (e.g., from Week 1 to Week 3) as shown in the process maps of learners in the control condition, and SRL-Q and SRL-QR prompt viewers (Fig. 4a, c, e).The process maps of learners in control condition (Fig. 4a) and SRL-Q prompt viewers (Fig. 4c) also show that there are more deviations from the sequential course weeks than the process map of SRL-QR viewers (Fig. 4e).Another interesting observation is that only in the process maps of SRL-QR prompt viewers (Fig. 4e), the paths show that learners return to previous course weeks only at the beginning (Week 2 → Week1) and at the end of the course (Week 9 → Week 8).

Discussion
In the second experiment, we examined the effect of the two types of SRL prompts in a nine-week MOOC.There was no significant support for any of the three main hypotheses (i.e., SRL-related activities, course engagement, and course performance) in the ITT analyses.However, for the TOT analysis, support was found for Hypotheses 1C and 1D: SRLprompt viewers in the SRL-QR condition completed a greater proportion of course activities on time and repeated a greater proportion of completed course items than the control group.This suggests that the SRL-prompt video consisting of three questions followed by recommendations benefitted learners by facilitating SRL activities indicative of time-management and self-reflection.The rest of the TOT analyses did not yield any significant results to support Hypothesis 2 (course engagement) and Hypothesis 3 (course performance), and hence, suggest that neither the SRL-prompt videos that provided questions nor SRL-prompt videos that provided questions followed by recommendations enhance course engagement and performance in the IM MOOC.
The exploration of the process maps to examine the sequences in which learners access the course items and completing them on time

Table 7
Means, standard deviations and trimmed (tr.) means for the conditions compared in the ITT analysis (based on active learners in all three conditions) and TOT analysis (comparing SRL-Prompt viewers in SRL-Q and SRL-QR conditions and active learners in the control condition) in IM MOOC.showed that SRL-QR prompt viewers had the highest average access to complete course items on time and the lowest percentage of learners who ended the process at Week 1. Also, SRL-QR prompt viewers access to complete course items appear to be more sustained across the course weeks.The dominant paths in the process maps that were visible at the 50% detail level suggest that there were less deviations from the sequential course weeks by SRL-QR prompt viewers (Fig. 4e).
The overall results of the IM MOOC seem to suggest a weak effect of SRL-QR prompts on SRL-related activities.Unlike in the SG MOOC, the SRL-Q prompts did not enhance learners' engagement nor performance in the IM MOOC.One reason could be the differences in effort needed to learn in the MOOCs.There were 46 course items distributed across the six weeks in the SG MOOC and graded assessments in every module whereas in the IM MOOC, there were 44 course items distributed across the nine-weeks in the IM MOOC and graded assessments only in alternate modules.The distribution of the types of course activities were also different in the MOOCs.Therefore, learning in the IM MOOC might not have been as effortful as in the SG MOOC (e.g., read text, take part in discussion, complete an assessment every week).These results together suggest that SRL-prompts may not be necessary to activate SRL activities that learners might need to succeed when faced with less effortful tasks.

Experiment 3: Econometrics
The third experiment was conducted in a MOOC on the topic of Econometrics: Methods and Application (EC).The EC MOOC is an eightweek MOOC on the topic of solving business and economic questions using data analytic tools and statistical models.One key topic was covered in each of the first six modules (e.g., simple regression, endogeneity).The seventh module consisted of only one course item in the form of a graded peer-reviewed assessment that required learners to apply the knowledge gained from the first six modules (i.e., weeks) of the course.The final week of the course consisted of optional topics that were labelled as building blocks with the purpose of providing learners with background knowledge of some concepts that are necessary to learn the other concepts taught in the MOOC.The recommended study pace for the EC MOOC was four to 8 h per week.Learning in the EC MOOC consisted of watching videos, downloading data sets, completing the training exercises, and reading the solution of the training exercises.To pass the EC MOOC, learners had to pass seven graded peer review assignments.Altogether, the EC MOOC had 129 course items distributed across the nine weeks.
The EC MOOC differed from the MOOCs in the previous two experiments in several ways.First, a module in the EC MOOC consisted of a main topic with a few subtopics.Each subtopic consisted of a video, a training exercise, and a solution.In the MOOCs of the previous two experiments, a module consisted of one main topic and the main topic was introduced by a series of videos concerning the main topic.Therefore, the EC MOOC appears to be more hands-on, requiring learning to work on a training exercise after each video.Another difference is in the type of graded assessment.The EC MOOC relied on only peer-review assignments whereas a mixture of quizzes and peer-review assignments were used in the SG and IM MOOCs.The third difference is in the recommended study pace.The EC MOOC called for the most time investment of the learners compared to the other two MOOCs.

Participants
As in the previous two experiments, data was collected over three cohorts of learners enrolled in the EC MOOC.Being one of the more popular MOOCs offered at the university, there were a total of 4493 learners who enrolled during the study period and they were randomly assigned to one of the three conditions (i.e., control, SRL-Q, SRL-QR).Table 9 shows the distribution of learners across the three conditions under each learner category (i.e., enrolees, active learners, and SRLprompt viewers) in the EC MOOC.In all three conditions, about half of the enrolees (51% in control, 50% in SRL-Q, and 54.2% in SRL-QR) went on to access at least one course activity (i.e., active learners).Within the SRL-Q and SRL-QR conditions, approximately half of the active learners (52.9% in SRL-Q and 48.8% in SRL-QR) watched at least one SRL-prompt video (i.e., SRL-prompt viewers).

Materials and procedure
The procedure of the experiment was the same as in Experiments 1 and 2. Learners were randomly assigned to one of three conditions (i.e., SRL-Q, SRL-QR, and control).After enrolment, learners were able to proceed with the course as they normally would and had access to all course materials.We embedded the SRL-prompt videos only in the modules covering learning content (i.e., the first six modules of the MOOC) and not in the week with graded peer-reviewed assignment (Week 7) and the week with optional topics (Week 8).Just like in the previous two experiments, the SRL-prompt videos in SRL-Q condition consisted of three questions each while the SRL-prompt videos in SRL-QR condition consisted of three questions followed by three recommendations each.

Measures and data analysis
The measures, data processing, and analytical procedures were similar to the previous two experiments.In Experiment 3, there were 140 completed responses in the motivation and SRL survey in Week 2 and 39 completed responses in the motivation and SRL survey in the final week of the MOOC.Kruskal-Wallis test was employed due to the violation of normality.Results show that learners in the three conditions at the beginning of the MOOC did not differ significantly in autonomous motivation, H (2) = 0.74, p = .69,and controlled motivation, H (2) = 1.35, p = .51.The means and standard deviations of the motivation and SRL subscales measured in Week 2 and the final week of the EC MOOC are reported in Appendix 2.

Results
The distribution of the number of learners across SRL-prompt videos watched in the EC MOOC is illustrated in Table 10.Means, standard deviations and 20% trimmed means for each of the outcome variables across the conditions compared are presented in Table 11.

SRL-related activities
For the ITT analysis, results showed that there was no significant effect of condition on learners' access of course preparatory items, F t (2, 992.56) = 2.39, p = .09,ξ = 0.06, proportion of course items completed on time, F t (2, 925.99) = 0.27, p = .76,ξ = 0.02, and proportion of completed course items that were repeated, F t (2, 927.23) = 0.06, p = .94,ξ = 0.01.For the number of visits to the grade information page, the standard error cannot be computed due to Winsorized variance of 0. Results from the Kruskal-Wallis test showed that there were no significant differences in the number of visits to grade information page across conditions, H (2) = 0.54, p = .76.
For the TOT analysis, a significant effect of condition was found for all SRL-related activities.First, there was a significant effect of condition on the number of access to course preparatory items, F t (2, 492.72) = 7.70, p < .001,ξ = 0.13.Post hoc tests revealed a significant difference between the control and SRL-QR conditions (95% CI [-4.09, − 0.96], p < .001),but not between the control and SRL-Q conditions (95% CI [-2.54, 0.32], p = .06)and also not between the SRL-Q and SRL-QR conditions (95% CI [-3.18, 0.35], p = .06).Second, there was a significant effect of condition on proportion of course items completed on time, F t (2, 432.4) = 23.32,p < .001,ξ = 0.21.Post hoc tests for the proportion of course items completed on time revealed significant differences between the control and SRL-Q conditions (95% CI [-0.02, − 0.01], p < .001)and between the control and SRL-QR conditions (95% CI [-0.03, − 0.01], p < .001),but not between the SRL-Q and SRL-QR conditions (95% CI [-0.02, 0.002], p = .05).Third, there was a significant effect of condition on the proportion of completed course items that were repeated, F t (2, 402.25) = 19.92,p < .001,ξ = 0.21.Post hoc tests for the proportion of completed course items that were repeated revealed significant differences between the control and SRL-Q conditions (95% CI [-0.01, − 0.002], p < .001)and between the control and SRL-QR conditions (95% CI [-0.01, − 0.003], p < .001),but not between the SRL-Q and SRL-QR conditions (95% CI [-0.01, 0.002], p = .28).Finally, for the number of visits to the grade information page, we applied a Kruskal-Wallis test and found a significant difference between the three conditions, H (2) = 6.56, p = .04.However, post hoc testing using Dunn's pairwise comparison with Bonferroni correction did not show any significant

Table 10
The distribution of number of learners across number of SRL-Prompt videos watched in the two SRL-Prompt conditions of the EC MOOC.

Table 11
Means, standard deviations and trimmed (tr.) means for the conditions compared in the ITT analysis (based on active learners in all three conditions) and TOT analysis (comparing SRL-Prompt viewers in the SRL-Q and SRL-QR conditions and active learners in the control condition) in the EC MOOC.differences between the control and SRL-Q conditions (p = .046),between the control and SRL-QR conditions (p = .33)and between the SRL-Q and SRL-QR conditions (p = 1.0).The results suggest that only SRLprompt viewers in the SRL-QR condition accessed more course preparatory materials than active learners in the control condition.However, both SRL-prompt viewers in SRL-Q and SRL-QR conditions completed more course activities on time and revisited more completed course activities than active learners in the control condition.

Course engagement
Results from the ITT analysis showed that there was no significant effect of condition on both the proportion of course items accessed, F t (2, 927.8) = 0.68, p = .50,ξ = 0.03, and the average number of course items accessed per active day in the course, F t (2, 926.43) = 0.01, p = .99,ξ = 0.01.In contrast, for the TOT analysis, a significant effect of condition was found for both the proportion of course items accessed, F t (2, 444.13) = 11.34,p < .001,ξ = 0.16, and the average number of course items accessed per active day in the course, F t (2, 485.54) = 8.08, p < .001,ξ = 0.12.
Corresponding post hoc tests in the TOT analysis for the proportion of course items accessed revealed significant differences between the control and SRL-Q conditions (95% CI [-0.02, − 0.001], p = .02)and between the control and SRL-QR conditions (95% CI [-0.03, − 0.01], p < .001),but not between the SRL-Q and SRL-QR conditions (95% CI [-0.02, 0.003], p = .05).Post hoc tests for differences in average number of access to course activity per active day in the course also revealed significant differences between the control and SRL-Q conditions (95% CI [-1.66, − 0.19], p = .01)and between the control and SRL-QR conditions (95% CI [-1.92, − 0.35], p = .002),but not between the SRL-Q and SRL-QR conditions (95% CI [-1.12, 0.70], p = .58).Results from the TOT analysis suggest that SRL-prompt viewers in both the SRL-Q and SRL-QR conditions are more engaged in the course than active learners in the control condition in terms of accessing the course activities available in the course to a greater extent and accessing more course activities on average for the days that they were active in the course.

Course performance
Although there were hundreds of learners in each of the conditions in the EC MOOC, very few learners obtained a course grade (Control, n = 26; SRL-Q, n = 32; SRL-QR, n = 27).We applied the Kruskal-Wallis test and found no significant difference in course grade obtained by learners in the three conditions for the ITT analysis, H (2) = 1.17, p = .56.For the TOT analysis, the Kruskal-Wallis test revealed s significant difference in course grade, H (2) = 11.97,p = .003.Pairwise comparison with adjusted p-values showed a significant difference between the control and SRL-Q conditions (p = .005),but not between the control and SRL-QR conditions (p = .046)and between the SRL-Q and SRL-QR conditions (p = 1.0).Therefore, the results suggest that learners in the SRL-Q condition who viewed the SRL-prompts obtained higher course grades than active learners in the control condition.

Process mining
Process mining maps for the five identified groups were created following the same steps in Experiments 1 and 2. The log data consisted 2325 active learners with 80923 access to course items.After processing the log data for access to complete course items on time, the log data consisted of 1860 active learners and 30136 access to complete course items on time.Table 12 shows the distribution of the learners and access to course items that were completed on time by the five identified groups.Fig. 5a-e are process maps illustrating the sequences of access to complete course items on time over the eight-week EC MOOC.Just like in the IM MOOC, the process maps with full details show "spaghetti-like" processes.Therefore, we set the paths at 50% detail to examine the processes of course items that were completed on time by the learners in the five groups.
The first observation from Table 12 is that the average access to complete course items on time is the highest for both SRL-Q and SRL-QR prompt viewers, followed by the control condition, and the SRL-Q and SRL-QR non-viewers.Next, we examined the frequency of access to complete course items on time across the eight-week EC MOOC by taking the course design of the EC MOOC into consideration: Week 7 consisted of only a case project in the form of a peer-review assignment and Week 8 consisted of supplementary topics that are necessary for learning the main concepts.In all process maps (Fig. 5a-e), the most number of access to complete course items on time is in Week 1.In process maps of the SRL-Q and SRL-QR non-viewers as well as SRL-Q viewers (Fig. 5b, d, c) there appears to be a higher frequency of access to complete course items on time in Week 8 as indicated by the darker shade of blue.The same shade of blue is observed in Week 2 of the SRL-Q viewers' process map (Fig. 5c), indicating more access to complete course items on time.The third observation is based on the number of learners who started and ended the process in Week 1.There is a high percentage of learners who started and ended the process in Week 1 across all groups.The highest percentage is observed in SRL-QR nonviewers (Fig. 5d and 93.26%), followed by SRL-Q non-viewers (Figs.5b and 84.31%), control condition (Figs.5a and 76.60%), SRL-Q viewers (Fig. 5b and 72.10%), and SRL-QR viewers (Figs.5e and 69.79%).
For the final observation, we examine the sequences of access to complete course items on time across weeks.As mentioned previously, Week 8 consisted of course items that provided learners with the prerequisite knowledge.Therefore, the first sequence of interest is the access to complete course items in the sequence of Week 1 → Week 8 →Week 1 to examine the number of learners who access course items in Week 8 before proceeding to learn the other concepts in the MOOC.By filtering the process maps for the sequence of interest, we found that 133 learners in total followed the sequence Week 1 → Week 8 →Week 1 (Control,n = 44;n = 4,n = 39;n = 3;n = 43).The results indicate that a greater proportion of SRL-Q and SRL-QR viewers access Week 8's course items before returning to Week 1 of course.The process maps of control, SRL-Q viewers and SRL-QR viewers as shown in Fig. 4a, c, and e suggest that at 50% level of detail, the paths taken by the learners are predominantly sequential but less so in SRL-Q and SRL-QR non-viewers.

Discussion
In the third experiment, the effect of the two types of SRL prompts was examined in an eight-week EC MOOC.The ITT analysis did not reveal any differences between the three conditions (i.e., control, SRL-Q, and SRL-QR) for all measured outcomes.In contrast, the TOT analysis revealed significant differences in all of the measured outcomes except for grade information page views.According to the TOT analysis, support was found for Hypothesis 1A (SRL-prompt viewers in the SRL-QR condition accessed course preparatory items more than active learners in the control condition), 1C, and 1D (SRL-prompt viewers in both the SRL-Q and SRL-QR conditions completed more course activities on time and revisited more course activities that were completed than active learners in the control condition).The results suggest that providing Average access to complete course on time 14.00 6.50 24.30 6.00 24.10 prompts followed by recommendations in the EC MOOC supported SRL activities related to planning, time-management, and self-reflection.Question-only prompts had a similar positive effect on timemanagement and self-reflection.
With regards to course engagement (Hypothesis 2), the TOT analysis showed that SRL-prompt viewers in both SRL-Q and SRL-QR conditions accessed the available course activities to a greater extent and on average accessed more course activities per active day in the course than active learners in the control condition.The results suggest that both types of SRL prompts have a positive effect on course engagement.Results on course performance showed that learners who viewed the SRL-Q prompts obtained higher course grades than active learners in the control condition.However, it is important to note that while the EC MOOC had the largest number of active learners of the three MOOCs examined, there were very few learners who obtained a non-zero course grade.
From the exploratory analysis of the process maps based on learners' access of course items completed on time, we observed that a higher proportion of learners in the SRL-QR and SRL-Q condition who viewed at least one SRL-prompt video accessed Week 8's course items to equip themselves with the prerequisite knowledge for the course before returning to Week 1 of the course.This can be considered as an SRL activity in which the learners plan for their learning by equipping themselves with the knowledge required for the course.We also observed that many learners started and ended the process in Week 1 across all groups.However, the percentage of SRL-QR prompt viewers who started and ended the process in Week 1 is the lowest.
The results in the EC MOOC suggest that both types of SRL prompts (i.e., questions only and questions followed by recommendations) benefitted learners in the EC MOOC as demonstrated by the higher SRL activities indicative of time-management and self-reflection as well as higher course engagement.Furthermore, SRL prompts in the form of questions followed by recommendations facilitated the access of course preparatory items indicative of planning.The results are aligned with previous studies that showed that SRL prompts enhance SRL-related activities (Sitzmann & Ely, 2010).

General discussion
The current study examined the effect of two types of prompts in three separate experiments on SRL activities, course engagement, and performance.The first type of prompt provided learners with three questions (SRL-Q) while the second type of prompt provided learners with three questions followed by three recommendations (SRL-QR).The prompts were presented to learners in the format of videos.Learners in    J. Wong et al. the SRL-Q and SRL-QR conditions had access to one SRL-prompt video each week to support the three phases of SRL according to Zimmerman's SRL model (i.e., planning, monitoring and reflecting).Table 13 summarises the significant findings from the three experiments.

SRL-related activities
The first research question relates to the effect of the two types of SRL prompts on SRL activities in MOOCs.The first SRL activity is related to planning.The TOT analyses show that learners in SRL-QR condition accessed the course preparatory items more than active learners in the control condition in the SG and EC MOOC but not in the IM MOOC.One possible reason could be that the effectiveness of the SRL support is influenced by the complexity of the MOOC (e.g., difficulty of content and number of course items).Lehmann et al. (2014) suggested that task characteristic, such as complexity, can have an impact on SRL.Both SG and EC MOOCs have graded assessments every week and the number of course items to be completed each week is on average higher than IM MOOC.Therefore, more planning might be required in SG and EC MOOCs than IM MOOC.There was no significant effect of SRL question only prompts on planning across all three MOOCs.One reason could be that the recommendations (e.g., Set clear learning goals on what you want to learn and make plans to achieve them.)were more direct in engaging learners in planning than question-prompts (e.g., Am I setting goals to ensure that I have a good understanding of the course materials?).Learners were explicitly told to make a plan or to have a plan.Therefore, such recommendations could have directed learners towards accessing the course materials as part of their planning.
The second SRL activity is related to one form of self-monitoring: checking one's progress in the course.We did not find any effect of the two types of SRL-prompts on the number of visits to the grade information page.One reason could be that visiting the grade information page is only one form of self-monitoring that learners engage in to judge their learning progress.The low number of learners who obtained a nonzero grade also suggests that completing graded assessments is not a typical activity of MOOC learners.Therefore, other ways of operationalizing monitoring in the context of MOOCs is needed to understand how learners monitor their learning in MOOCs.
The third SRL activity is related to time-management.Time-management is an area of challenge for MOOC learners (Alario-Hoyos, Estévez-Ayres, Pérez-Sanagustín, Kloos, & Fernández-Panadero, 2017).Our results show that both SRL-Q and SRL-QR prompt viewers in the SG and EC MOOCs completed a higher proportion of course items on time than the control condition.However, in the IM MOOC, only SRL-QR prompt viewers completed a higher proportion of course items on time than the control condition.The results suggest a consistent positive effect of SRL prompt questions followed by recommendations on time management.One possible reason for the general positive effect could be that some of the recommendations included in the SRL-prompts specifically prompted time-management activities (e.g., pace yourself, set time-management goals).Therefore, in all three MOOCs, learners benefitted from the taking on the recommendations.Another reason could be the complexity of the MOOCs.As mentioned previously, it could be that both SG and EC MOOCs required more effortful to complete.Therefore, learners in the SG and EC MOOCs are more aware of the challenges to stay on track, and hence, benefitted from both types of prompts.
The fourth SRL activity related to self-reflection.The results showed that SRL-QR prompt viewers in both the IM and EC MOOCs and SRL-Q prompt viewers in the EC MOOC had a higher proportion of completed course items that were repeated than the control condition.One of the differences between the SG MOOC and the IM and EC MOOCs is the presence of discussion prompts.In the SG MOOC, there was a discussion prompt each week to help learners to elaborate on the concepts learned (e.g., Give a definition of Serious Game).These discussion prompts were complementary to some of the questions and recommendations provided in the SRL prompts such as "At the end of your learning session, think about what you have learned in the course for this week.".Therefore, it could be that discussion prompts are already supporting self-reflection in the SG MOOC across all conditions.Similarly, in IM MOOC the discussion prompts were only present in Weeks 1 and 6.In the EC MOOC, there were no discussion prompts.Therefore, it appears to be helpful to provide learners in MOOCs that have minimal or no discussions that reinforce learning with SRL-prompts in the form of questions followed by recommendations to support the self-reflective process and enhance reviewing of course items.

Course engagement and performance
In terms of course engagement and performance, we found positive effects of the SRL prompts only in the SG and EC MOOCs.SRL-Q prompt viewers in both the SG and EC MOOCs completed the course items available in the courses to a greater extent and had a higher average access of course items per active day in the course than the control condition.These results align with Sitzmann and Ely's (2010) study which showed that questions prompting SRL enhance course engagement.Compared to IM MOOC, SG and EC MOOC required considerably more effort to learn.In complex learning environments, learners often fail to spontaneously self-regulate their learning (Azevedo, 2009;Bannert & Mengelkamp, 2013).One of the reasons is the lack of awareness of the gap between their current state of learning and the desired state of learning.Therefore, the questions prompting SRL (e.g.Am I concentrating on learning the materials in this course?)can remind learners to employ SRL activities (e.g., find a quiet place to focus on completing the course items) that would otherwise not be spontaneously employed.Results also showed a positive effect of SRL-prompts consisting of questions followed by recommendations in the EC MOOC.
Compared to the SG MOOC, the EC MOOC was a lot more challenging and required more time investment.The topic in the EC MOOC was also harder to understand without prerequisite knowledge.It is possible that providing learners with SRL recommendations after the question prompts are helpful for learners in such MOOCs who need more directed support to self-regulate their learning.Besides higher course engagement, SRL-Q prompt viewers in the EC and SG MOOCs also obtained a higher course grade.However, our results on course grades should be interpreted with caution due to the low number of learners who obtained a non-zero course grade.The results contradicted studies which showed that supporting SRL enhances course completion and performance (Bannert & Reimann, 2012;Jansen et al., 2020).One of the reasons could be that we used final course grade as the only indicator of course performance in MOOCs, and by doing so, we captured only one aspect of course performance.Not all learners have the intention to pass all graded assessments or earn a course certificate (Henderikx, Kreijns, & Kalz, 2017).Therefore, future studies can explore other ways of operationalizing course performance, such as goal intentions.
Our study also contributes to the fields of SRL and learning analytics by employing process mining to explore the sequences of learners' access to course items to complete them on time.The process mining maps offer insights into the frequency of access to complete course items on time, the proportion of learners who started and ended the learning process in Week 1, and the sequential pattern in which learners access the course items to complete them on time.In all three MOOCs, SRL-Q and SRL-QR prompt viewers had a high number of access to complete course items on time.Also, the percentage of SRL-Q and SRL-QR prompt viewers who started and ended their learning process in Week 1 was lower than for learners in the control condition.In the EC MOOC, we observed that a greater proportion of SRL-Q and SRL-QR prompt viewers took a path that is indicative of planning (Week 1 → Week 8 → Week1).In general, there is a more linear pattern following the course weeks for the SRL-Q and SRL-QR prompt viewers with less skipping of course weeks or returning to previous course weeks.The observed pattern aligns with Wong et al.'s (2019b) finding that show that learners complete course items in a sequential manner.However, it is important to note that the results of the process mining are descriptive.In addition, we only explored one perspective (i.e., sequences of access to complete course items on time) and examined the sequences across course weeks.To get an even more detailed picture of the learning process, future studies can examine the sequences by course items or take the instructional design of the MOOC into account.

Limitations
Conducting randomized controlled trials in MOOCs can be rather challenging given that learners are not only in control of the pace of learning but they also have the autonomy to interact with as little or as much course content as they want.The first limitation is the selfselection bias.MOOC learners in the study had the choice of viewing or not viewing the SRL-prompt videos.It can be argued that learners who viewed the SRL-prompt videos are the more active learners compared to the learners in the control group.However, when compared to the control condition, results differed between learners who viewed the SRL-prompt videos in the SRL-Q and SRL-QR conditions.Therefore, the positive findings in the study cannot be due to selfselection bias alone but also the effect of the type of prompts.Future studies should continue to include a third comparison group or an active control group to reduce the self-selection bias in MOOC studies.
The second limitation is the compliance to the SRL-prompt (i.e., watching the SRL-prompt videos).The current study identified SRLprompt viewers in the SRL-Q and SRL-QR conditions as learners who viewed at least one SRL-prompt video.While the results showed that around 30%-50% of the learners fall into the category of SRL-prompt viewers across all MOOCs, this category included learners who could have watched as little as one SRL-prompt video (i.e., low compliance) or as many as all the SRL-prompt videos (i.e., high compliance) that they had access to.The results from our study are aligned with previous studies (Davis et al., 2016;Jansen et al., 2020), suggesting that learners' interaction with the SRL prompts is a critical factor.To better understand the effect of the SRL-supports on learners' SRL activities, engagement, and performance, future studies are encouraged to take learners' extent of interaction with the SRL-support (e.g., the number of SRL-prompt videos viewed by learners) into account.
The third limitation of our study is the operationalization of outcome variables measured in the study.In the current study, we identified four types of behaviors from the log data to indicate SRL activities and we operationalized course engagement by the number of unique access to the course activities that are available in the course.By doing so, we made assumptions about SRL and engagement in MOOCs.Particularly for the SRL activities, some of the indicators as measured from the log data may not be as directly linked to the actual SRL process as others (e. g., time management as indicated by the proportion of course items completed on time compared to self-monitoring as indicated by the number of visits to the grade information page).We also acknowledged that some of these SRL activities can be operationalized by more than one indicator from the log data, for example, visiting grade information page is only one form of self-monitoring and the process of selfmonitoring can be manifested in other ways depending on what is possible in the learning environment (e.g., checking ones' answers in a quiz).Similarly, course performance was only defined by the course grades.Learning successes in traditional courses defined by grades, passing rate, and completion of course activities might not necessarily apply to MOOC learners who are taking the courses for fun or for curiosity.Therefore, future studies examining learning in MOOCs can expand the operationalization of SRL activities, course engagement and learning outcome using other methods of data analysis and other sources of data.For example, studies can examine course engagement by examining the number of study sessions that learners make.Future studies can also account for the diverse goals that MOOC learners might have (e.g., complete only the first two modules) to examine whether learners' goals influence the effectiveness of SRL prompts.

Conclusion
Supporting self-regulated learning (SRL) is an important endeavour to support learners in MOOCs, and in turn, enhance learners' success in MOOCs.The autonomy that learners have in MOOCs adds a layer of complication to examining the effects of SRL-prompt videos.Building on studies that support SRL in MOOCs, our study showed that prompting SRL in the form of questions as well as questions followed by recommendations facilitate self-regulatory learning activities as well as course engagement.However, the results of the three experiments are not consistent, suggesting that the effectiveness of the type of prompts may be influenced by the complexity of the MOOCs as well as the characteristics of the learners (Lehmann et al., 2014).Future studies should continue to build on these findings and examine whether the effect of the two types of prompts can be replicated in other MOOCs and to better understand the factors that influenced the effectiveness of the SRL prompts.

Appendix 3
Results of non-parametric Kruskal-Wallis test and corresponding pairwise comparison to supplement main analysis of robust ANOVA with 20% trimmed means across the three experiments.

Fig. 2 .
Fig. 2. Overview of the number of interactions with course items (i.e., events) and number of learners (i.e., cases) shown in the Disco software.Top figure shows interactions of all active learners in the SG MOOC.Bottom figure shows only learners' interactions with the course items that were completed on time.

Fig. 3 .
Fig. 3. a, Process map of learners in control condition completing course items on time in the SG MOOC, 3b.Process map of SRL-Q prompt non-viewers completing course items on time in the SG MOOC, 3c.Process map of SRL-Q prompt viewers completing course items on time in the SG MOOC, 3d.Process map of SRL-QR prompt non-viewers completing course items on time in the SG MOOC, 3e.Process map of SRL-QR prompt viewers completing course items on time in the SG MOOC.

Fig. 4 .
Fig. 4. a, Process map of learners in control condition completing course items on time in the IM MOOC, 4b.Process map of SRL-Q prompt non-viewers completing course items on time in the IM MOOC, 4c.Process map of SRL-Q prompt viewers completing course items on time in the IM MOOC, 4d.Process map of SRL-QR prompt non-viewers completing course items on time in the IM MOOC, 4e.Process map of SRL-QR prompt viewers completing course items on time in IM MOOC.

Fig. 5 .
Fig. 5. a, Process map of learners in control condition completing course items on time in the EC MOOC, 5b.Process map of SRL-Q prompt non-viewers completing course items on time in the EC MOOC, 5c.Process map of SRL-Q prompt viewers completing course items on time in the EC MOOC, 5d.Process map of SRL-QR prompt non-viewers completing course items on time in the EC MOOC, 5e.Process map of SRL-QR prompt viewers completing course items on time in the EC MOOC.
J.Wong et al.

Table 1
Number of learners assigned across the three conditions in each learner category in the SG MOOC.

Table 2
The Distribution of Number of SRL-Prompt Videos Watched in the two SRL-Prompt Conditions in SG MOOC.

Table 4
Distribution of learners who completed at least one course item on time and the frequency of access to the course items completed on time in SG MOOC.
in terms of accessing course preparatory materials.However, unlike Jansen et al.'s (

Table 5
Number of learners assigned across the three conditions in each learner category in the IM MOOC.

Table 6
The distribution of number of learners across number of SRL-Prompt videos watched in the two SRL-Prompt conditions of IM MOOC.

Table 8
Distribution of learners who completed at least one course item on time and the number of access to the course items completed on time in the IM MOOC.

Table 9
Number of learners assigned across the three conditions in each learner category in the EC MOOC.

Table 12
Distribution of learners who completed at least one course item on time and the number of access to the course items completed on time in the EC MOOC.

Table 13
Summary of significant findings of comparisons in the ITT and TOT analyses across the three MOOCs.

Table 2 .2
Means and Standard Deviations on the Learners' Motivation and Self-Regulated Learning at Week 2 and Final Week of IM MOOC Across the Three Conditions

Table 2 .3
Means and Standard Deviations on the Learners' Motivation and Self-Regulated Learning at Week 2 and Final Week of EC MOOC Across the Three Conditions

Table 3 .1
Results of Kruskal-Wallis Test and Dunn's Pairwise Comparison Test with Bonferroni Correction to Follow Up on Significant Results of Kruskal-Wallis Test for Experiment 1 on SG MOOC