The impact of student misconceptions on student persistence in a MOOC

Massive Online Open Courses (MOOCs) provide opportunities to learn a vast range of subjects. Because MOOCs are open to anyone with computer access and rarely have prerequisite requirements, the range of student backgrounds can be far more varied than in conventional classroom-based courses. Prior studies have shown that misconceptions have a huge impact on students' learning performance; however, no study has empirically examined the relationship between misconceptions and learning persistence. This study of 12,913 MOOC-takers examines how students' misconceptions about the upcoming course material affect course completion. Using a survival analysis approach, we found that, controlling for the score in a pre-course test, students holding more misconceptions had a higher dropout rate at the start of the course, an effect that diminished over time. Other student variables were found to have a positive impact on survival that persisted throughout the entire course: U.S. location, higher age, an intention to complete, better English skills, prior familiarity with the subject, motivation to earn a certificate, and score and time spent on the previous problem set (homework). By contrast, student gender, education level, number of previous MOOCs completed, and motivation to participate in online discussion forums did not affect survival.

about the upcoming course material affect course completion.Using a survival analysis approach, we found that, controlling for the score in a pre-course test, students holding more misconceptions had a higher dropout rate at the start of the course, an effect that diminished over time.Other student variables were found to have a positive impact on survival that persisted throughout the entire course: U.S. location, higher age, an intention to complete, better English skills, prior familiarity with the subject, motivation to earn a certificate, and score and time spent on the previous problem set (homework).By contrast, student gender, education level, number of previous MOOCs completed, and motivation to participate in online discussion forums did not affect survival.
A lack of motivation among participants is considered to be the key reason for the high dropout rate (Khalil & Ebner, 2014;Pursel, Zhang, Jablokow, Choi, & Velegol, 2016;Shapiro et al., 2017;Xu & Yang, 2016).Belanger and Thornton (2013) identified distinct motivations of MOOC learners, such as participating for life-long learning, for fun, for convenience, or for the experience.Such motivations further influence student self-regulated engagement patterns, such as being completers, samplers, or no-shows (Hill, 2013;Kizilcec, Piech, & Schneider, 2013;Wilkowski, Deutsch, & Russell, 2014).Recently, scholars have considered motivation to be a component of student self-regulation (Barak, Watted, & Haick, 2016;Magen-Nagar & Cohen, 2017).Ryan and Deci (2000) proposed a self-determination theory (SDT), which posits that learners need a sense of autonomy, aptitude, and relatedness to stay engaged (Durksen, Chu, Ahmad, Radil, & Daniels, 2016;Hartnett, George, & Dron, 2014;Ryan & Deci, 2000).Other researchers have applied the expectancy-value theory to the learning motivation in MOOCs, whereby motivation has been defined as a function of one's expected chance of success, perceived usefulness of the course and the estimation of the cost (e.g., De Barba, Kennedy, & Ainley, 2016).As shown by Reich (2014), only 22% of those who claimed to be strongly motivated to finish the course actually finished.This suggests that motivation is not constant, but adjusts based on students' course experiences and time investment (Hone & El Said, 2016).As students have just started enrolling in the course, their inadequate background and competence with the subject matter often reduce their self-efficacy (Shapiro et al., 2017), which in turn downgrades their motivation to complete the course (Sawtelle, Brewe, & Kramer, 2012).Chen, Sonnert, and Sader (2019) have shown a salient MOOC engagement pattern in which learners went to the final assessment when they are still at the beginning of the course.Those with higher prior competence passed the assessment, gained self-efficacy, and stayed in the course; and those who failed the assessment lost their confidence and expedited their dropout.
A great amount of MOOC research has focused on the competence, or readiness, of the students (Breslow et al., 2013;Greene, Oswald, & Pomerantz, 2015;Kizilcec & Halawa, 2015;Milligan, Littlejohn, & Margaryan, 2013).In most of this literature, competence has been measured by self-reported familiarity, prior experience (Breslow et al., 2013;Greene et al., 2015;Kizilcec & Halawa, 2015;Milligan et al., 2013), or a general skill as a proxy (Chen et al., 2019), but most of these measures are problematic when applied to participants with very limited experience, a novice population precisely targeted by entry level MOOCs.Although such students may not have knowledge about the specific course content, they nevertheless hold various preconceptions consolidated from life experience (Fisher, 1985) or convenient model representations (Chen, Schneps, & Sonnert, 2016;Gentner & Wolff, 2000).
Knowledge of student misconceptions has long been considered a crucial element of a teacher's skill (Baumert et al., 2010;Ergönenç, Neumann, & Fischer, 2014;Keller, Neumann, & Fischer, 2017;Sadler et al., 2013).It is an important component of pedagogical content knowledge (PCK), a term created by Shulman (1986) and expanded by generations of researchers in teacher knowledge (e.g., Ball & Bass, 2000;Grossman, 1990;Magnusson, Krajcik, & Borko, 1999).Indeed, Sadler et al. (2013) have shown that students cannot correct their misconceptions by themselves over time, even if their teachers have solid subject matter knowledge, only students whose teachers have knowledge of student misconceptions can achieve conceptual change.

| Misconceptions and dropout
No study, to the best of our knowledge, has linked misconceptions and student dropout (or retention).In traditional college classroom settings, where the time scale is long (counted in semesters or years) and the dropout penalty is huge (losing tuition, credits, or even the opportunity to earn a degree) (Mortagy, Boghikian-Whitby, & Helou, 2018), dropping out because of a beginner's psychological frustration during mental paradigm shift may be considered extremely costly.Studies have shown that the frustration induced by cognitive and emotional conflict may lead to students' psychological burnout-exhaustion or cynicism (Khalaj & Savoji, 2018;Olwage & Mostert, 2014;Salanova, Schaufeli, Martínez, & Bresó, 2010), or disengagement (Gan, Shang, & Zhang, 2007;Maslach, Jackson, Leiter, Schaufeli, & Schwab, 1986).Burnout, however, has been found to materialize as dropout only when the frustration becomes severe (Bask & Salmela-Aro, 2013;Duque, 2014;Ensminger & Slusarcick, 1992).Specifically, Salanova et al. (2010) showed that students' burnout was strongly predicted by a classroom setting that combined the presence of learning obstacles with the absence of facilitators-an environment similar to the traditional MOOC setting.Moreover, in the MOOC setting, where the time scale for taking the course is short (ranging between weeks and hours, in the extreme), and the dropout penalty is minimal, a moment of cognitive conflict may justify dropping out of the course.
Thus, though MOOCs differ from offline learning contexts, they qualify as a "strategic research site" (Merton, 1987) for three reasons.First, MOOCs an expanding and ever more important format for science education.Second, they provide the opportunity to examine how students' retention is affected by their prior misconceptions, an effect that is otherwise too small to detect in traditional classroom settings.This, in turn, facilitates a more comprehensive understanding of the effects of misconceptions.Third, it may allow inferences for the domains of out-of-school-time and informal science learning where dropout is similarly easy, but where learners' characteristics and behaviors can rarely be tracked in the comprehensive way that MOOCs afford.
Although no prior study, to the best of our knowledge, has empirically examined the relationship between misconceptions and retention, science learning theories have implied a relationship between the two.In the following, we examine this pattern through the theoretical perspectives of cognitive conflict theory and expectancy-value theory.
Many studies have investigated how misconceptions evoked cognitive conflict, related to student-teacher dynamics (Larkin, 2012;Nixon et al., 2016;Sadler et al., 2013).Traditionally, researchers assumed that learners hold coherent and theory-like mental models, like scientists would, which implied that learners should first be dissatisfied with their existing conceptions in order to be motivated to acquire new knowledge and generate more plausible and coherent theories-a concept that lies at the core of the classical cognitive conflict approach (e.g., Hewson & Thorley, 1989;Ioannides & Vosniadou, 2002).Recent studies have presented evidence that students do not always hold coherent mental models, but rather scattered elements from multiple perspectives because for them, as novices, their responses are highly context-dependent (Bao & Redish, 2006).This finding gave rise to more gradual and contextualized pedagogies-known as the cognitive perturbation approach (Dega, Kriek, & Mogese, 2013;Li, Law, & Lui, 2006;Özdemir & Clark, 2007).Overall, both approaches have stressed that cognitive conflict is a valuable window for motivating inquiry for deep understanding (Appleton, 2008;Chow & Treagust, 2013;Delgado & Lucero, 2015;Halim & Meerah, 2002;Harmon-Jones, Amodio, & Harmon-Jones, 2009;Larkin, 2012;Sadler et al., 2013;Treagust & Duit, 2008;Van Driel, Verloop, & De Vos, 1998).
Although cognitive conflict has often been considered to be a valuable opportunity and premise for conceptual change, it inevitably frustrates the students (Dega et al., 2013).Cognitive conflict often occurs together with (De Dreu & Weingart, 2003;Simons & Peterson, 2000), and even provokes (Kellermanns & Floyd, 2005;Mooney, Holahan, & Amason, 2007), affective (emotional) conflict.This problem has been raised as early as 1979 by Carl Frankenstein, who worried that a lengthy experience with cognitive conflict may increase students' frustration so much so that they halt conflict resolution.This is especially true for low academic achieving students.Zohar and Aharon-Kravetsky (2005) showed that cognitive-conflict-inducing pedagogy was effective only for high achieving students and hindered the progress of low achieving students.
This dual function of cognitive conflict is also discussed in the framework of threshold concepts, which are "bottle-neck" or "gate-keeping" concepts in one's knowledge progression that open a new perspective that was previously inaccessible and invite the learners to irreversibly transform their mental model (Meyer & Land, 2006).However, when learners perceive the threshold concepts to be counter-intuitive, intellectually absurd, and emotionally frustrating, they may revert back to, and get stuck with, their original misconceptions by alienating the new concepts, which was what Perkins (1999Perkins ( , 2006) ) referred to as troublesome knowledge.
A common assumption embedded in the abovementioned conceptual change literature is that students would keep learning in the course regardless of whether they were retaining or revising their intuitive misconceptions.No component in the conceptual change theories has explicitly proposed or modeled the possibility that students with strong misconceptions are likely to become motivated, or resigned, in the early stages of an intervention when they encounter cognitive conflict.As argued above, the conceptions that coexist in a student's mind are often conflicting beliefs (Lawson, 1988;Potvin & Cyr, 2017;Potvin, Sauriol, & Riopel, 2015;Smith, 1994).Students who hold strong and systematic misconceptions tend to find the new knowledge system taught to them in formal learning settings to be counterintuitive (Guzzetti, 2000).
It is possible that learners are motivated to resolve this cognitive conflict by learning more course content (as reviewed earlier, cognitive conflict motivates students to question and to learn), which would predict longer retention for learners with more misconceptions than for learners with fewer misconceptions (provided that their total amount of subject matter knowledge are the same).For example, in a hypothetical scenario, two learners respond to 10 subjectmatter questions before taking a (MOOC) course and both answered 5 questions wrong.If out of the 5 wrong answers, learner A gave 3 misconception responses, and learner B gave 0 misconception responses, it is predicted that learner A would have longer retention in the course than would learner B; or conversely, that learner B would dropout earlier from the course than would learner A, because learner A would be more intrigued to learn more to resolve the cognitive conflict.
However, despite of the possibility of cognitive-conflict-induced curiosity, learners may also need extra efforts to inhibit their intuitions (Foisy et al., 2015;Masson et al., 2014) and to restructure their mental models (Gadgil, Nokes-Malach, & Chi, 2012;She, 2004;Vosniadou, 1991).The mismatch between one's intuition and the taught subject matter knowledge, and the extra efforts required to reconcile the mismatch, may frustrate students and increase their resignation.Such resignation may explain why misconceptions are stubborn, and many intervention efforts, futile.It may also predict that learners with more misconceptions have lower retention than do learners with fewer misconceptions.According to this hypothesis, learner A is predicted to drop out earlier than learner B in the above hypothetical scenario.
Thus, from cognitive conflict theory, one might deduce opposing hypotheses about the effect of misconception on retention.It might be positive or negative.In the online learning environment, in particular, it is difficult for teachers to update their knowledge of students' misconceptions interactively, and also difficult for them to monitor students' cognitive conflict.As explained above, a lack of knowledge of student misconception among teachers, combined with a lengthy cognitive conflict experienced by students, may not effectively promote conceptual change, but may exacerbate students' struggle and resignation.Therefore, we were more inclined to hypothesize that misconceptions have negative effects on retention.It is of theoretical importance to empirically examine the effects and to discern between the two mutually opposite hypothesis.Moreover, this is not a problem that only concerns the online learning environment.As research has shown (Pintrich, Marx, & Boyle, 1993;Zohar & Aharon-Kravetsky, 2005), any classroom in which the teacher is not sensitive to students' misconceptions or cognitive conflict may go through the same struggle, but that struggle may not manifest itself as openly as in the MOOCs context.
Cognitive conflict theory also implies that the effect of misconception on retention should diminish over time as the misconceptions are resolved or inhibited.As learners with many misconceptions update their mental model, and the course content appears less counter-intuitive, intellectually absurd or emotionally frustrating, they are expected to persist in the rest of the course on equal footing with those learners with initially fewer misconceptions.

| Expectancy-value theory
One of the most-cited theories in examining dropout behavior is expectancy-value theory.It has been widely applied to explain why people drop out from careers (e.g., Luscombe, Lewis, & Biggs, 2013), from school (e.g., Fan & Wolters, 2014), from science interests (e.g., Sullins, Hernandez, Fuller, & Tashiro, 1995), and from MOOCs (e.g., De Barba et al., 2016).Developed by Atkinson (1957) and through multiple generations of modification (e.g., Eccles & Wigfield, 1995), the expectancy-value theory posits one's motivation in a task to be a function of (a) the expectancy of success (based on an estimation of the task difficulty), (b) the utility value of task completion, and (c) the estimated cost (money, effort or time) of achieving the expected success.
In Atkinson's original version of the expectancy-value theory, expectancy was tied to the success rate in the task that an individual has experienced in the past.Other researchers theorized that expectancy was formulated not only based on prior success but also on the evaluation of the task difficulty and one's self-competence (e.g., Eccles & Wigfield, 1995).One crucial assumption behind both formulations of expectancy was that people know the success and failure of their past experience or, in the context of knowledge acquisition, have an approximately accurate evaluation of what they know or do not know.Nevertheless, the fact that many elements of the existing knowledge people hold are misconceptions that are confidently believed to be true (and successful in explaining a lot of the past experience) seriously challenged the assumption that one can accurately estimate the expectancy of success, or the assumption that people in general have about the same insight into their past success rate, their current competence and their future performance.A person who answers 5 questions wrong and gives 0 misconception responses may be more aware of his/her lack of subject matter knowledge than a person who answers 5 questions wrong, but gives 3 misconception responses.In other words, learners with more misconceptions may be overly optimistic about their past success and their expectancy of success.This optimism may be a buffer that prevents them from dropping early out of a MOOC, as prior studies have shown that over-optimism is preferable to over-pessimism in helping learners persist in science subjects (Bench, Lench, Miner, Flores, & Liew, 2015;Watt, 2010).However, once the courses kick off, learners holding more misconceptions may soon perceive stronger mismatch between their expectancy and the actual task difficulty than may those who have similar amount of subject matter knowledge, but fewer misconceptions.This mismatch may inform the learners of their miscalculation of the expectancy and cast doubt on their self-concept and self-efficacy, which may negatively influence their course retention.Thus, expectancy-value theory also implies two opposite hypotheses about the relationship between misconception and retention: over-optimism may have a positive effect on retention; or the disturbed expectancy may have a negative effect on retention.The expectancy-value theory also implies that once a learner adjusts his/her expectation over time, the effect of misconception should diminish gradually.

| Hypothesis
We hypothesized that holding misconceptions may be negatively or positively associated with retention in the initial stages of a MOOC, but not in intermediate or later stages.The reasoning was that the mismatch should be most strongly felt when students carrying misconceptions were first introduced to the new and scientific knowledge system, but that, as the students adapted to the new system, they should persist equally well as students with fewer prior misconceptions.This should manifest as an interaction effect of misconception and course milestones on the probability of dropout at a corresponding milestone.This hypothesis was supported by earlier work by Chen et al. (2019) that showed precomputational thinking skills, a measure of logical and algorithmic thinking styles prevalent in computer science, to have a positive impact on students' retention in an introduction to computer programming MOOC.However, that effect diminished to non-significance over the course milestones.The cited study, however, measured competence, using a general skill as a proxy; it did not measure misconceptions.In our current study, we will extend our understanding of MOOC retention by connecting it specifically to students' misconceptions.
In this study, we used year 2017 data about students' characteristics, activities, and performance in the MOOC Super-Earths and Life (SPU30x), which was a HarvardX MOOC, available on the edX platform.The course was taught by a professor of astronomy.The course used astronomy and space science concepts to discuss the discovery of exoplanets (planets around other stars) that could be favorable for life.
The course has been offered on the edX platform since 2015, each year the teaching team makes very minor revisions to the course content.Although the teaching team was not oblivious of the notion of misconceptions, it did not intentionally address students' misconceptions in the design of the course content and pedagogy.The principal investigators of this study collaborated with the teaching team in data collection, but the investigators and the teaching team were independent from each other.Thus, SPU30X should be considered a regular astronomy and space science course, not a special treatment for astronomical misconceptions.
The major chapters of the course included (a) reviewing the Earth in the solar system, with an emphasis on the spatial, chemical, and climate conditions that make life possible (4 sessions); (b) measuring the distance to the planets and stars, measuring their mass and size, and making inferences about their formation (4 sessions), (c) understanding the types of exoplanets, plate tectonics, and atmosphere on exoplanets (5 sessions), and (d) detecting signals of life, using telescope and spectrometer, and wrap-up (5 sessions).If each session is considered a milestone, there were 18 milestones.

| Learning astronomy
When the journal Science asked about the most exciting open questions of science, "Are we alone" ranked at the top of the list (Kennedy, 2005), which is also one of the few science questions that are equally appealing to both genders (Krstovic, Brown, Chacko, & Trinh, 2008).The search of exoplanets and alien life forms is an ideal topic that attracts learners who wish to learn out of curiosity (social capital) rather than for professional skills (human capital).Yet, such a topic connects to core and crosscutting concepts from multiple disciplines (Gould, Sunbury, & Dussault, 2014;Rijsdijk, 2000).Thus, science education practitioners have created many resources and curricula that teach about exoplanets online, and considered it to be one of the best practices of online learning for the promotion of science literacy of the public (Gould, Dussault, & Sadler, 2006;Gould, Sunbury, & Krumhansl, 2012).
Although astronomy is fascinating to a broad population, it is also one of the science subjects that people most rapidly lose interest and do not pursue at a deeper level of understanding (Bergstrom, Sadler, & Sonnert, 2016;Sadler, Sonnert, Hazari, & Tai, 2012).Studies have shown that learners often prefer to stay at a superficial and misconceptual understanding of astronomy (Bailey & Slater, 2003;Snyder, 2000) even if they were introduced to scientifically accurate conceptions (Champagne et al., 1985;Chi, 2005;Gilbert, 2004).From a learning progression point of view, the understanding of the complexity of astronomy depends on grasping steppingstone concepts (also known as threshold concepts) and overcoming intuitive misconceptions (also known as troublesome concepts).Sadler (1992Sadler ( , 1996) ) showed that grade 8-12 students failed to understand the reason for day and night because they believed that the Earth orbits the Sun in a day.Further, with a confusion about orbiting and spinning, it is nearly impossible for learners to understand the galactic rotation, using spectroscopy.For another example, once learners understand the scales of the distances between Earth, the planets, and the stars, astrology would not make sense to them (Sadler, 1996).Conversely, if students do not resolve scale-related misconceptions, they will, in addition, remain troubled in grasping many other concepts, such as the change of the seasons (Trumper, 2001) and the phases of the Moon (Plummer, 2006), which all rely on understanding scale (Fanetti, 2001;Miller & Brewer, 2010).Chen, Chudzicki, et al. (2016) have shown that, when astronomy learners switch from scale-accurate models to scale-exaggerated models, they can keep acquiring new knowledge without developing scale-exaggeration-related misconceptions; however, when learners switch from scale-exaggerated models to scale-accurate models, the misconceptions associated with the scale exaggeration remain strong, and learners do not acquire any new knowledge from inspecting the new and more accurate models, as if the learners mentally shutdown (or burnout) from receiving more accurate, yet cognitive-load-heavy information once they consolidate with the attractive misconceptions.
The abovementioned studies and others that showed that misconceptions are tenacious only argued that misconceptions can reduce learning, but did not contemplate the possibility that learners mentally stopped learning altogether when they found the new knowledge to be incongruent with their existing perspectives, because they were forced to sit in the classrooms.In a MOOC setting, students can voluntarily quit whenever they want, which enabled us to explore the relationship between preexisting misconceptions and actually quitting learning.This study of the relationship between misconceptions and retention may inform about how learners withdraw from further knowledge acquisition under the influence of preexisting misconceptions.A study of such a topic may be valuable to the field of astronomy education, and also to the broader field of science education, and especially in the domain of out-of-school-time and informal science learning.
The advantage of choosing an astronomy topic for this study was that (a) it has been well studied that novices have naïve misconceptions about basic astronomy concepts, such as the change of seasons, the phases of the moon, or cosmic scales (Ashmann, 2012;Barrier, 2010;Comins, 1998;Turkmen, 2017;Zeilik & Morris, 2003); (b) it is known that misconceptions about such fundamental concepts in astronomy have long standing deleterious effects on students' learning of space science in formal classes (Trumper, 2001;Zeilik et al., 1998); and (c) there are well developed misconceptions-oriented test banks in astronomy for novices, tests that have good psychometric properties (e.g., Eryilmaz, 2002;Sadler et al., 2013) and they are public available (Sadler et al., 2010).Therefore, SPU30X is not only a topic that is appealing to a broad population, but also presented a subject field in which the effect of misconceptions is strong, well-documented, and convenient to replicate.

| Research question
Thus, our research question was whether the number of student misconceptions in the space science background knowledge test (misconception score) from the presurvey (pretest) was associated with the dropout rate at a given milestone, and if its effect interacts with the number of milestones that have been completed.We hypothesized an initial effect (increasing the dropout rate) that would attenuate over the course of the MOOC.We adopted a survival analysis approach to investigate this relationship.In this model, we controlled for the total score in the pretest as well as other covariates, such as students' demographic information, motivation, prior experience, the time elapsed since passing the most recent previous milestone, and their grade in the problem set (pset) of the most recent previous milestone (as explained below in the variable section).

| Sample and baseline variables
Thirty thousand six hundred ninety-six individuals registered for the MOOC SPU30X on edX; however, only 12,913 of those registered came back to the course and finished the presurvey, which was a prerequisite to gain access to the course material.Nine hundred and thirty-eight of those finished the presurvey did not continue viewing the course, which reduced our analytical sample to 11,966.In this article, we considered those who finished the presurvey as formal enrollees and applied statistical analysis only to the formal enrollees.Around 9% of the participants were so-called "samplers," meaning they skipped at least one milestone in their sequence (e.g., someone could complete problem sets [psets]1, 2, and 5, and then drop out, skipping pset-3 and pset-4).This irregular pattern is not suitable for a survival analysis framework and was investigated in a separate study.Here, we excluded the irregular participants, which reduced our sample size to 10,014.

| Presurvey
Among the 10,014 enrollees, 40% were male, 60% were female.The mean age was 29.5 years (SD = 11.2),63% were living in a country outside of the USA.Thirty-four percent of the enrollees were concurrently going to school, and 53% had a college or higher degree.On average, enrollees had registered in 1.6 MOOCs and had completed 1.2 MOOCs prior to this MOOC enrollment.Familiarity with the topic of the course was reported by 23%; a somewhat or strong motivation to earn certification by 52%; and a somewhat or strong motivation to participate in the online forum by 26%.Eighty-eight percent reported being proficient or fluent in English.
The presurvey included a space science background knowledge test (the pretest) drawn from items deemed relevant to the concepts covered in the course and derived from the Astronomy and Space Science Concept Inventory Project (Sadler et al., 2010).The test items were multiplechoice questions about space-related science that were chosen from the existing validated test bank that covered knowledge required by the National Science Education Standards (National Research Council, 1996) and the American Association for the Advancement of Science Benchmarks for Science Literacy (Project 2061(Project , 2001)).The development of the items and associated answer choices was guided by existing research on learning trajectories of key concepts in astronomy and space science and by the way these concepts were represented in national standards (Plummer & Krajcik, 2010;Plummer & Maynard, 2014;Sadler, 1996).
Each test item contained four choices from which students were to select, with one correct choice, one attractive misconception choice (chosen by more than 50% of the participants in the large scale field test who answered the item incorrectly), and three plain wrong (or less distractive) answers that did not reflect popular misconceptions.Because each item has one misconception as a distractor, this type of test is known as a misconception-driven test.The development of the items went through several key steps, including (a) literature review, (b) writing draft items, (c) expert validation, (d) pilot test, (e) large scale field test, (f) psychometric analysis, (g) constructing final test items, and (h) practice testing the final test items.The complete inventory (211 items) has a Cronbach internal reliability of 0.85.In the field test carried out in 2003, high school students (the target population who the items were designed for) correctly answered 50% of the items, on average.The corresponding percentages were 65% for college students and higher than 80% for school science teachers (see Sadler et al., 2010, for detailed procedures of test development and for psychometric properties of the items).
The pretest used in this study comprised 12 items (see Appendix).The average sum score for correct answers was 7.95 (standard deviation: 2.28), and the average misconception score was 2.64 (standard deviation: 1.75).Below is an example item from the pretest.It probes for concepts about the source of energy and the scale of energy produced by the stars.The correct answer is (c), and the most common misconception is (d), which is rooted in combustion ideas, which, in turn, lead to failures in estimating the magnitude of energy produced by stars.
An astronomer would say that most stars produce energy in the same way as: Regarding MOOC performance, the course contained 18 milestones, and each milestone ended with a problem set (pset).On average, participants completed 7.53 problem psets out of the total number of 18 psets.Three thousand fifty-one (23% of all) participants finished all 18 psets, in line with the completion rate in previous years (~20%).On average, students spent 47 min on each milestone session (SD = 39).The questions included in the psets simply revisited the course content; they were very easy, with an average difficulty of 0.1 (only 10% learners gave wrong answers).Thus, the psets mostly served as a check point, not necessarily probing learners' advanced knowledge.

| Analysis
To model dropout rate at a given milestone as a function of predictors (such as pretest, motivation, etc.), we adopted a survival analysis approach.A survival analysis involves three basic terms: event, time, and censoring.In our case, event is student dropout (1 = dropout; 0 = completion) at a given milestone, time is the course milestone, and censoring is if a subject does not experience dropout during the whole MOOC period (in other words, the student completes all milestones).Survival analysis is analogous to logistic regression: the dropout event is a binary outcome variable; milestone and other covariates are predictors; and the model parameters can be interrelated in the same fashion as a logistic regression.
As basic steps for survival analysis (see Singer & Willett, 2003), we first calculated the hazard for each milestone period.The hazard function represents the proportion of each milestone interval set that dropped out during that interval: where h(m ij ) is known as the population discrete-time hazard.M i represents the milestone period j when individual i experiences the dropout event (e.g., for a student who drops out at the third milestone, M i = 3).The hazard function denotes that the probability that the dropout event will occur at a certain milestone j for student i is conditional on student i not experiencing the dropout event at any time prior to j. Table 1 showed the life table for the observed sample, the survival, cumulative failure and hazard function at each milestone interval.
Next, we used a logit link function to link the hazard to a linear specification of predictors, similar to a logistic regression: In this function, M i and M i 2 together represent the linear and quadratic main effect of a milestone.There are multiple possible specifications of the main effect of a milestone, such as treating milestones as dummies (if the hazard function has an irregular form) or as a linear main effect (if the hazard function is close to a linear line).Upon inspection of the logit hazard function, we decided that a quadratic specification would parsimoniously and accurately reflect the hazard function in our case.Predictors of interests in this model are the X variables and U variables.X variables are time invariant variables; they include students' age, gender, motivation, familiarity, pretest score, English fluency, foreign status, etc. (see Table 1 for the full list of variables).Such variables were only measured in the initial questionnaire (milestone 1).They reflected students' initial status and were considered time invariant.U variables are timevarying predictors.In our case, there were three time-varying predictors, which were the score in the previous pset (variable name: pscore), and time spent in the previous milestone (variable name: active_time).We used pscore and active_time as a proxy for students' performance and engagement, but the validity of such usage was arguable.Both variables were time-lagged by one milestone from the dropout event to be predicted.One reason was that when participants dropped at a milestone, their pset scores would be missing, and their active_times in the milestone of the dropout event would be extremely low or missing.
The time-lagged model allowed us to predict the odds of dropout in the upcoming milestone, based on the performance in the most recent active milestone.The course contained four chapters; therefore, there were three occasions for new-chapter = 1, respectively at milestone 5, 9, and 14, whereas new-chapter = 0 for other milestones.Thus, new-chapter could be considered as a discrete time-varying predictor.If we found the estimated coefficient of new-chapter to be positive and statistically significant, we would conclude that the dropout event was more likely to happen after the end of a chapter and before the beginning of a new chapter.
The parameters (βs and γs) associated with the Xs and Us stand for the shift in the baseline logit hazard function (as depicted by the main effect of a milestone), corresponding to unit differences in the associated predictors.In other words, the logit hazard function of students with different X or U values shift up and down, but the shape of the function should be identical as it is determined by the main effect of a milestone when we do not take into consideration the interaction terms.We also considered interaction terms between predictors and milestones.This allowed different students to have different shapes of the logit hazard function, depending on their Xs and Us.When two groups (categorized by a predictor of interest such as gender) have converging logit hazard curves, it means the two groups have a larger difference in dropout rates at the beginning, and that this difference decreases over time (i.e., the effect of the group predictor attenuates).If the logit hazard curves diverge between two groups, it means the group differences increase over time.We used a post-GLM test to examine if and at which milestone the two logit hazard curves converge or diverge.

| RESULTS
Table 2 presents the parameters for the fitted model.The model used quartic terms to define the relationship between dropout and milestone.The predictors included time constant variables, such as age, gender, prior familiarity, which were collected only once (in the presurvey) and were considered to be invariant over time.The model also included time-varying predictors, such as students' performance in the psets and time spent on the milestone, which varied at each milestone.For ease of interpretation, we converted the estimated parameters of the final model to odds ratios by exponentiation and then to marginal probabilities (the change of the probability of dropping out corresponding to a one-unit change in a specific covariate, provided that other covariates are held constant at their means and that the milestone is held at milestone 1).Interpretation of the parameters is analogous to the interpretation of a logistic model: β shows the amount of change in logit hazard associated to one unit of change in the predictor, and the logit hazard can be converted to an odds ratio.For example, β located_in_US = 0.26, which shows that the logit hazard for participants located in the US was larger by 0.26 than the logit hazard for participants located outside of the US, controlling for other covariates.This could further translate to an odds ratio of 1.30 (e 0.26 = 1.30), which means that the odds of dropping out for a US-local participant were 1.30 times as high as the odds of dropping out for an outside-of-the-US participant.Controlling the other covariates at their means and assuming milestone = 1 and new-chapter = 0, we calculate the marginal probability (comparing probability when located_in_US = 1 versus located_in_US = 0) of dropout was 0.063, which means that, at milestone 1, the probability of a US-local student dropping out was 6.3% higher than that of an outside-of-the-US student, if every other covariate was controlled at its mean.This difference was statistically significant.
Similarly, students who reported having stronger intentions to earn a certificate, students from outside of the United States, students of older age, students who reported being more familiar with astronomy, and students who reported having better English skills, had lower odds of dropout at each milestone, compared with their counterparts in the respective reference categories.These predictors did not have an interaction effect with milestones, which means changes to these predictors were associated with the elevation of the fitted line (or curve) of the logit hazard curve but did not change the slope of the line (or the shape of the curve).By contrast, there was a significant interaction effect between misconception score and the milestones: students who had a higher misconception score-controlling for the total scorehad higher odds of dropping out at the initial milestones, but this effect diminished over time.This interaction relationship is illustrated in Figure 1, which plotted two prototypical groups that had misconception score scores ±1 standard deviation of the mean, while keeping all other covariates at their means (the variable new-chapter was kept at zero).According to post-GLM test (χ 2 = 0.94, p = .33),the two groups were not statistically significantly different from each other starting from milestone 7, which was in the middle of the course chapter that discussed measuring the distance, mass and size of exoplanets.
The model also contained time-varying predictors related to participants' in-class performances, respectively, the score in the previous pset (pscore) and the time spent in the previous milestone session (active_time).We found both pscore and active_time to have significant main effects.This indicated that, regardless of which milestone students had progressed to, the lower their scores were the pset, and the less time they spent on the milestone, the higher were their odds of dropout in the following milestone.
In summary, this study had three main results.First, we discovered several predictors of dropout whose effects did not diminish over time (i.e., remained constant throughout the milestone sequence): location, age, intention to complete, English skills, prior familiarity to the subject, and motivation to earn a certificate, score and time spent in the previous pset.Second, we did not find gender, education level, number of previous MOOCs completed, and motivation to participate in online forums to have significant effects on the likelihood of dropout after controlling for other variables.Third, our finding rejected the hypothesis that preexisting misconceptions have a positive motivating effect on course retention, and it supported the competing hypothesis that misconceptions predict dropout.Moreover, the effect of misconceptions was strong at the beginning, but diminished over the course of the milestone sequence.

| DISCUSSION
We first discuss the implications of our findings for MOOCs research and practice and then focus on their contribution to the misconception literature in traditional classroom settings.
The most important finding of this study was the effect of the misconception score, which had an interaction effect with milestones.This result confirmed a prior study (Chen et al., 2019) that has shown that prior acquaintance within the knowledge domain had a positive effect on persistence and that this effect decreased as students progressed through the milestones.The test for preexisting knowledge adopted by the prior study was an aptitude test to examine students' algorithmic and logic skills before learning computer programming.The pretest in this study was the first test, to our knowledge, that measured students' misconceptions in a MOOC study.Therefore, it provided the first evidence to show that misconceptions influence students' dropout rates in MOOCs.Specifically, misconceptions pose an initial hurdle to participation.When the concepts covered by the course contradict the misconceptions held in students' intuition, students might find the course content difficult to grasp early on.
The conceptual domains that were probed by the pretest (scale, energy, spectroscopy, and multi-factorial mental models) were fundamental to the understanding of the movement of spatial objects, the distance between the objects, the transfer of energy, and the observation of signals, which are the building blocks and reoccurring topics in the SPU30X.If misconceptions were not addressed in these domains, they would become troublesome knowledge that would block the learner not only from deep understanding but also from further inquiry.Here are a few examples: 1. a learner who assumes that the Sun produces energy by burning oxygen might expect that it would burn out over the course of a few thousand years; 2. a learner who assumes that our solar system was solely created by the Big Bang might find it ridiculous that Earth contains elements generated in supernovae; 3. a learner who holds an over-simplified notion of scale might underestimate the challenge presented by the vast distances in space exploration; and 4. a learner who holds misconceptions in spectroscopy, such as believing that all stars are white, could not make sense of how scientists make inferences about distant objects primarily based on observing the light from them.
We did not posit any particular misconceptions to trigger a learning obstacle at any particular course milestone, which is a challenge beyond the scope of this study.Our pretest sampled a limited number of misconceptions that learners may hold in their mental models.Our results did show that, controlling for the level of astronomy knowledge, holding larger numbers of misconceptions constitutes an additional obstacle to a learner's persistence in the course, suggesting that multiple encounters of a counterintuitive mismatch between the course contents and the learner's existing conceptions exacerbate psychological burnout and the inclination to withdraw from further learning.As we discussed in the theoretical framework, the mismatch may activate three immediate reactions of the learners: cognitive conflict, disturbed expectation, plus frustration as a result of the two.This study did not explicitly measure any of the three possible reactions, but they may serve to explain the findings of the study from a theoretical perspective by positing a plausible mechanism.
The results also helped us decide between opposing predictions of the theories and thus update our understanding of these theories.If, according to cognitive conflict theory, cognitive conflict did occur, it was likely not motivating learners to resolve the conflict by persisting in the course, but it rather discouraged them from further inquiry, perhaps to avoid the discomfort or to ease the burnout of cognitive conflict.If, as implied by expectancy-value theory, preexisting misconceptions did lead to over-optimism about the learners' existing knowledge, it appears that the over-optimism did not help them to persist as it was found in other STEM contexts, such as career interests (Bench et al., 2015), but that it rather frustrated the learners.Whereas this study could not determine which of the two theoretical frameworks was at work, it was able to conclude that holding distractive misconceptions was worse than lacking subject matter knowledge (recalling that we had controlled for the learners' overall level of correctness in our model, those students who held neither the scientific knowledge nor a popular misconception were considered to have a lack of knowledge, or to hold idiosyncratic incomplete or erroneous beliefs-because few learners have a complete lack of knowledge) because the misconceptions would drive the learners astray and constitute an additional penalty to learners' persistence, at least in the initially stages of a MOOC setting.
Interestingly, our study also showed that, as students kept participating in the course through the initial chapters, possibly by picking up increasing levels of content knowledge, resolving/inhibiting their misconceptions, adjusting their expectations and self-evaluations, and/or managing their frustrations, they would persist as well as those who had fewer misconceptions at the outset.We do not know if the learners had successfully resolved their misconceptions (as it has been proven very difficult to achieve), or if the resolved misconceptions were the reason that ameliorated the steep dropout trend.It is possible that learners may have gradually ignored their misconceptions (as misconception inhibition was found to be more common than misconception resolution), or managed their frustration, or adapted their estimation of the course difficulty and adjusted their learning effort (psychological cost) devoted to the course.Alternatively, our findings could be simply attributable to the possibility that those who felt the strongest discomfort about cognitive conflict or disturbed expectation had already dropped out in the initial stages, and those who stayed were not bothered by the discomfort.All these possible explanations call for more targeted studies in the future, as we will elaborate in the Limitation section.
Our findings lead to three major suggestions to astronomy MOOC instructors, or to MOOC instructors in general, if we assume generalizability beyond astronomy (see Section 5).First, MOOC instructors should make efforts to measure and understand students' incoming knowledge, including misconceptions.It has been well documented that knowledge of students' misconceptions (part of the PCK) is a crucial teaching skill for teachers to facilitate their students' learning in traditional classrooms (Hill et al., 2005;Sadler et al., 2013); and such knowledge has proven to be useful in online settings as well (She & Liao, 2010;Wendt & Rockinson-Szapkiw, 2014).This study suggests that teachers should not only pay attention to how students' misconceptions may affect their understanding and performance, but also how it may impede students from engaging in understanding and performing at all.
Second, MOOC instructors should actively address students' misconceptions.Our original hypothesis that misconceptions may motivate learners to engage in more inquiry was based on the premise that learners have teachers or resources to turn to when their existing mental models were challenged.A common principle behind various cognitive-conflict-driven pedagogical approaches is that teachers should be aware of the students' misconceptions so that they can purposefully use students' cognitive dissonances as opportunities to either explicitly resolve the cognitive conflict (Bucat, 2015;Wartono & Putirulan, 2018), or promote the scientific explanation to surpass the misconception while they still coexist in the mind of the students (Potvin et al., 2015).Sadler et al. (2013) have shown that students cannot correct their misconceptions by themselves over time, and only students whose teachers have knowledge of student misconceptions can achieve conceptual change.In the absence of such facilitators, cognitive conflict may lead to frustration, and educational opportunities may be missed.One of the common and major shortcomings of the MOOC platform is the lack of customized attention and scaffolding.To address learners' misconceptions individually may be costly for most MOOC platforms; however, considering the importance of misconceptions for students' persistence, instructors should at least anticipate the most common misconceptions in the topic field and allocate time to address these misconceptions in the initial stages of the course.
Last, from an expectancy-value theory perspective, instructors should be aware that misconceptions may seriously bias students' evaluation of self-competence and the expectation of success, as learners who hold strong misconceptions may assume they already have a working mental model or a good understanding of some of the content.Such an optimism is not sustainable once learners start the course and realize what they are learning is way more difficult and frustrating to grasp than they had expected.Instructors should help the learners set their expectation in the beginning of the course.For example, the instructors can review the pre-screening test with the learners, inform them about their misconceptions, and/or preview the expected learning curve of the course.Most importantly, the instructors should inform the learners that, as the learners persist in the course, they are expected to perceive smaller and smaller amounts of frustration induced by counterintuitive subject matter knowledge.
As noted in the Introduction, the general public, especially young learners, are naturally fascinated by astronomy or science in general, but this fascination often gives way to the mystification of science-believing that science is awe-inspiring, yet understandable only by a small group of geniuses (Dimopoulos & Koulaidis, 2003;Evans, Krippendorf, Jae, Posluszny, & Thomas, 1990), settling on superficial knowledge and misconceptions, and preventing people from reconstructing their mental models to accommodate new knowledge.A lot of prior discussion in the misconception literature addressed how misconceptions make learning difficult.We argue that it is an even more troublesome and alarming problem in science education when misconceptions make learning stop completely in the very beginning stages.Our findings inspired us to ask a more philosophical question: is counter-intuitive knowledge hard to assimilate because it is difficult to resolve even if people take the time to attempt resolutions, or because people do not even take the time to wonder?This was precisely what Carl Frankenstein (1979) worried about 40 years ago-that cognitive conflict might increase learners' frustration so much that they halt conflict resolution.
This philosophical speculation assumes an analogy between MOOC engagement and general science learning engagement, which is an open question.As pointed out in the introduction, dropping out from class or from school is always a costly decision in the traditional educational system.In some situations, for example, if a class is elective, students can drop out within a few trial sessions, but the dropout rate is usually very low because of academic, financial, and peer pressures.Even though, as has been shown in prior studies, students may experience multiple moments of temporary defeat, or burnout, due to cognitive conflict or frustration (Khalaj & Savoji, 2018;Olwage & Mostert, 2014;Salanova et al., 2010), students rarely can afford to actually drop out.Therefore, the variation in attrition that might be explained by misconception is very small, which makes the traditional school setting impractical for studying the relationship between misconception and attrition.In the MOOCs setting, students can drop out at any time with very little cost, which is ideal for investigating factors influencing attrition.It is noteworthy that MOOCs are very different from traditional classes in many ways.For example, students were found to have higher satisfaction levels and higher learning outcomes in traditional classrooms than in an online environment (Smith, Wilson, Banks, Zhu, & Varma-Nelson, 2014).Wendt and Rockinson-Szapkiw (2014) further showed that online collaborative activities were less effective than in-classroom ones in addressing student misconceptions.Nevertheless, the construction of knowledge should follow similar progressions, and students should experience similar hurdles when they first encounter and acquire new concepts that are contrary to their prior beliefs.Based on the result of this study, we speculate that, in traditional classroom settings, students with prior misconceptions are prone to increased feelings of failure at the beginning of the class.This speculation appears to be difficult to test empirically in classrooms without extensive student observations, interviews, or surveys about their psychological states.However, we further predict, based on a weaker version of our speculation, that in an elective offline course that allows students to drop in a "trial period," students who drop out in this period should have stronger prior and unresolved misconceptions, compared with students who persisted, controlling for equivalent scores in pre-tests.This prediction should be easy to test in future studies and would strengthen the analogy between MOOC engagement and classroom engagement.Earlier research (e.g., Brobst, Markworth, Tasker, & Ohana, 2017;Coe, Aloisi, Higgins, & Major, 2014;Sadler et al., 2013) has shown that teachers who understand students' misconceptions tend to be high quality teachers and help students improve their grades in traditional classrooms.Part of this effect, as we further speculate, could be attributed to the phenomenon discussed in this article: as students' misconceptions have been addressed, their frustration, thoughts of failure and intentions to give up, are eased.We suggest the reader be aware of the untested analogy between dropping out of a MOOC (and informal science learning activities) and the psychological resignation in the classroom.It is possible that our finding is only applicable to the MOOC settings (and perhaps the informal science learning domain).Future studies should investigate the interplay between teachers' perceptions of students' misconceptions, teachers' pedagogies, and students' misconceptions, frustration, resilience, and performance, in both online and offline classrooms.
A MOOC such as SPU30X, which teaches about the search for Earth-like planets and alien life, is fundamentally different from MOOCs that teach immediately useful skills or tools, such as computer programming or statistics.In comparison, SPU30X participants were more likely to be driven by personal interests-like hobbyists-rather than driven by occupational skill development.Compared, for instance, with the MOOC CS50x Introduction to Computer Science, the content in SPU30X contains more narratives, similar to educational documentaries, and is much less demanding of prior mathematic, logic, or language skills.The completion rate of SPU30X (23%) is above the average completion rates (15%, ranging between 5% and 40%) reported in the MOOC literature (Jordan, 2015).Thus, this study contributes to MOOC research by its coverage of a hobbyist MOOC, a type of MOOC less examined by researchers.Yet, for the same reason, caution should be taken when generalizing the results of this study to other types of MOOCs.Nevertheless, by successfully replicating the effects of many covariates that have been well documented by the studies of other MOOCs, it appears plausible that SPU30X was not overly different from the general MOOCs family after all.
Another limitation of the study was that we did not keep track of the misconceptions over the course, which limited our ability to examine the mechanism behind the association between misconceptions and retention.Had we directly measured learners' misconceptions repeatedly over the milestones (such as including misconception measures in the psets, instead of using the actual psets that only revisited course content and had very low difficulty), we could inspect if the misconceptions were gradually resolved and if the change in misconceptions was associated with the change in dropout hazard.Lastly, we could not explicitly discuss what it means that the probability of dropout converged at around milestone 7 between the high and low misconception groups.Milestone 7 was in the middle of the second chapter that discussed the measurement of distance, mass and size.It may be strongly related to misconceptions in scale and spectroscopy that we measured in the pretest.However, we do not suggest there to be a precisely aligned relationship between the content in the chapter and the measured misconceptions for two reasons: First, scale and spectroscopy concepts were not only applied to chapter two.They have been introduced in the first chapter that discussed the position and environment of Earth that made life possible, and also repeatedly applied in later chapters about observing features of the exoplanets.Second, the misconceptions included in the presurvey served to collect a sample of learners' misconceptions and were not comprehensive enough to diagnose the exact domains of misconceptions the learners held.Thus, a higher misconception score should be interpreted as having more misconceptions about astronomy in general.In short, this study cannot detect the specific misconceptions that interacted with the course content at specific milestones, because we did not cover milestone-specific misconceptions in the pretest and only measured misconceptions that were widely applicable to most of the milestones.

| CONCLUSION
To our knowledge, this is the first study to show students' misconceptions to be an obstacle to persistence in the initial sessions of a MOOC.We also found students' performance and engagement in the most recent milestone to predict their persistence in the following milestone.These findings have very clear policy implications for improving the design and teaching of the next generation MOOCs.
a. a wood fire.b. molten rock.c. a hydrogen bomb.d. a chemical reaction between two gases.e. a welding torch.

F
I G U R E 1 Plotting the fitted probability of dropout by misconception score levels (mean ± 1 SD) with 95% confidence intervals [Color figure can be viewed at wileyonlinelibrary.com] Life table with the observed sample and the survival, the cumulative failure and the hazard function at each milestone interval T A B L E 1 Survival analysis predicting dropout from MOOC Super-Earths and Life T A B L E 2Note: *p < .05;**p < .01;***p < .001,after false discovery rate (FDR) adjustment.