Research Trends in Massive Open Online Course (MOOC) Theses and Dissertations: Surfing the Tsunami Wave

Massive Open Online Courses (MOOCs) have attracted a great deal of attention by higher education and private enterprises. MOOCs have evolved considerably since their emergence in 2008, all the while given rise to academic discussions on MOOC impact, design and reach. In an effort to understand MOOCs more comprehensively, this study analyzes theses and dissertations (N = 51) related to MOOCs and published between 2008 and 2015, identifying research trends from these academic documents. Theses and dissertations within this research scope were gathered through a comprehensive search in multiple academic databases. For the purposes of the study, the research employed a systematic review approach. In order to reveal trends in research themes, emphasize theoretical/conceptual backgrounds, research designs and models, first a document analysis was used to collect data and this was followed by a content analysis. Our research findings indicate that MOOC research is generally derived from education, engineering and computer science, as well as information and communication technology related disciplines. Qualitative methodology linked to a case study research model is most common, and the theoretical/conceptual backgrounds are usually distance education related. Remarkably, nearly half of the studies didn’t benefit from any theoretical or conceptual perspectives. In sum, this study presents an evaluation regarding research trends derived from MOOC theses and dissertations, and provides directions for future MOOC research.


Introduction
, the year 2008 was a cornerstone for networked learning and MOOCs. Dave Cormier first coined the term MOOC to define connectivist learning on networks (Hollands & Tirthali, 2014). George Siemens and Stephen Downes facilitated the first (connectivist) MOOC in that same year (Siemens, 2013). Many other successful connectivist MOOCs followed one another. As a connected and open system, MOOCs caught a lot attention and were researched in open and distance education (de Waard et al., 2011, Bozkurt et al., 2015a. However, MOOC mania really took off when Sebastian Thurn facilitated the Artificial Intelligence (AI) MOOC which attracted 160K learners from 190 countries in 2011. This meant that the first generation connectivist MOOCs (cMOOCs), suddenly saw the rise of a second MOOC generation called extended MOOCs Open Praxis, vol. 8 issue 3, July-September 2016, pp. 203-221 (xMOOCs). The rise of xMOOCs seemed unstoppable. In an article in the New Yorker, the President of Stanford, John Hennessy said, "There's a tsunami coming" (Auletta, 2012, p. 71) and Daphne Koller, a professor of Computer Science at Stanford University and the co-founder of Coursera, responded by saying "The tsunami is coming whether we like it or not. . . You can be crushed or you can surf and it is better to surf" (McKenna, 2012, para.2). The authors of this paper believe that in order to surf on the MOOC tsunami, there is a need to map current state of art in MOOC research. In this regard, the purpose of this research is to present research trends emerging from MOOC theses and dissertations published from 2008 up to 2015.

MOOC Hype: It is real and here
Although MOOCs were first introduced in 2008, scientific knowledge about MOOCs is still at an early stage. Understanding why, how and through what processes MOOCs are used and by what means they affect their users, is still a big challenge. Being such a new phenomenon, one must wonder if all this hype is generated from their substantial contribution to the intellectual development, or if it is a result of a promise of new emerging technologies (Johansson & Frolov, 2014). In order to understand MOOCs' maturity, adoption and application, each key event or development regarding MOOCs is identified and spotted by the authors Gartner Hype Cycle of key MOOC events ( Figure 1). Key events such as Open Educational Practices, Open Educational Resources and first MOOCs were identified in Technology Trigger phase. First commercial MOOC platforms and the Year of the MOOC (Pappano, 2012) were identified in Peak of Inflated Expectations phase. Declaration of Anti-MOOC year (Watters, 2013) was identified in Trough of Disillusionment phase and described as sliding into the trough, while developments such as new type of hybrid MOOCs (Ross, Sinclair, Knox & Macleod, 2014;Bozkurt & Aydin, 2015) and increasing MOOC research (Liyanagunawardena, Adams & Williams, 2013;Gasevic, Kovanovic, Joksimovic & Siemens, 2014;Sa'don, Alias & Ohshima, 2014;Ebben & Murphy, 2014;Veletsianos & Shepherdson, 2015;Raffaghelli, Cucchiara & Persico, 2015) were identified in Trough of Disillusionment phase and described as Climbing the Slope. The year 2015 was spotted at the beginning of the Plateau of Productivity. MOOCs as a catalyst for higher education and MOOC evaluation were identified in Slope of Enlightenment phase. Plateau of Productivity phase wasn't identified with any of the developments or events, yet considering the rapid progress of MOOCs, emerging business models, increasing educational adoption by higher education and millions of registered lifelong learners in pursuit of knowledge, it is believed that MOOC phenomenon is at the verge of the Plateau of Productivity phase and will enter this phase in near future.

Figure 1: Gartner Hype Cycle of key MOOC events/developments
This means that in contrast to criticisms and negative predictions made in 2013, interest in MOOCs is now evolving at an unprecedented pace, fueled by the attention given to high profile entrants like Coursera, Udacity, and edX in the popular press (Johnson et al., 2013). Besides, there is an increasing interest from lifelong learners, higher education institutions and for-profit platforms from 2012 onwards, which is also visualized in the Google trends for MOOCs ( Figure 2). This pattern is quite similar to those reported by Liyanagunawardena et al. (2013), Ebben andMurphy (2014), Sa'don et al. (2014) and Raffaghelli et al. (2015). Since 2012, MOOC pedagogy has gotten increased attention with clear indications for its big impact on lifelong learning as concluded in the consequent Innovation Reports from the Open University, UK (Sharples et al., 2012;2014).

State of the art of MOOC research trends
As an emerging phenomenon in online learning, MOOCs are like a massive research laboratory (Diver & Martinez, 2015). However, there is still a large gap in MOOC research (Saadatmand & Kumpulainen, 2014;Sinclair, Boyatt, Rocks & Joy, 2014;Loizzo, 2015). This has multiple reasons such as data protection concerns, a tendency to hoard data conspire to curtail data sharing, ethical hurdles for qualitative research (Esposito, 2012;Reich, 2015). In this regard, there is a need for a coherent research agenda (McAuley, Stewart, Siemens & Cormier, 2010) and in order to understand how we should design and develop learning for the future, we must first take stock of what we know and what has been well researched (Siemens, Gasevic & Dawson, 2015).
Currently the state of the art of MOOC research trends is as follows: • The first study concerning MOOC research trends belongs to Liyanagunawardena et al. (2013) who conducted a systematic review of the published MOOC literature (2008)(2009)(2010)(2011)(2012) and examined 45 articles. The above mentioned articles covered research papers, articles and proposals, however research on MOOCs should not be limited to these academic documents, which is where this study steps in.

Significance and rationale of the study
Grey academic literature, in this case theses and dissertations, are another important source of information to use while examining research trends. Theses are written for an MA degree and dissertations are written for a PhD degree. They are formal and lengthy scholarly publications required at the end of MA or PhD programs to earn these degrees. Further, theses and dissertations have a role as a means of creating, distributing and disseminating scientific information (Bozkurt et al., 2015b). Within this perspective-as a complementary study-this research intends to contribute to better understand MOOC literature. Understanding a discipline-namely MOOCs-on macro as well as micro levels, is partially possible through examining changes, dynamics, and perspectives in theory and practice in the research conducted in that particular field (Bozkurt et al., 2015b). It is also important to create a research agenda based on research trends to guide individual researchers, institutional groups, and regional, national and international agencies, associations, and other networks (Anderson & Zawacki-Richter, 2014) because understanding factors and the new dynamics in the field provides future researchers and practitioners with a comprehensive scheme of experiences, implications, practices, policies, programs, and perspectives.

Aim of the Study
This study analyzes theses and dissertations related to MOOCs and identify research trends by examining theses and dissertations in 2008-2015 period. By making the research trends visible, research gaps can be located, and future research can be planned to enable the shift from the Slope of Enlightenment to the Plateau of Productivity. Within this perspective, the research question for this study is as follows: What are the most employed research trends in theses and dissertations published between 2008-2015, while looking at research methods, research themes, research models/designs, as well as theories, concepts, frameworks and models.

Methodology
This follows a qualitative research design and uses a systematic review. Systematic reviews aim to identify, appraise and summarize studies of a particular topic (Webb & Roe, 2007) and are used to arrive at a more holistic, comprehensive and trustworthy outlook of the research topic (Gough, Oliver & Thomas, 2012). Systematic reviews provide guidance to researchers planning future studies, and provide convenient summaries of the literature on a particular issue (Petticrew & Roberts, 2008). In current MOOC systematic review research, the data collection and organization process employs document analysis while data analysis process employs content analysis approach. Document analysis, is a social research method. It is an important research tool in its own right and is an invaluable part of most schemes of triangulation employed for data collection. In document analysis, different techniques can be used depending on the type of the document and purpose of the research (Heffernan, n.d.). Document analysis involves skimming (superficial examination), reading (thorough examination), and interpretation. This iterative process combines elements of content analysis and thematic analysis (Bowen, 2009).
Content analysis is not aligned to one methodological approach (e.g., qualitative or quantitative) and can be based on a variety of data types (Banks, Louie, & Einerson, 2000). It is a research method for making replicable and valid inferences from data to their context, with the purpose of providing knowledge, new insights, a representation of facts and a practical guide to action (Krippendorff, 1980). The aim is to attain a condensed and broad description of the phenomenon, and the outcome of the analysis is concepts or categories describing the phenomenon. Usually the purpose of those concepts or categories is to build up a model, conceptual system, conceptual map or categories (Elo & Kyngäs, 2008). Considering the aim of the study, summative content analysis is designated as an appropriate analysis approach to review and interpret MOOC research.

Sampling, Limitations and Strengths
This research sampled theses and dissertations published between 2008 and 2015 to reveal research trends. The authors employed purposeful sampling. The findings of the study are limited to theses and dissertations written in English, with online, full text accessibility, published from 2008 to 2015, and having one of the following definitions in the title: MOOC(s), Massive Open Online Course(s), cMOOC and xMOOC. The following databases were used: Google Scholar, ProQuest, Open Access Theses and Dissertations, WorldCat, Anadolu University database, Athabasca University database and Open University database. 51 documents met the sampling criteria and were included in this research (Appendix 1). This study's strength is that it is adding a base study for future research because it is the first study which examines theses and dissertations to identify trends in MOOC research.

Data Collecting, Procedure and Analysis
The 51 theses and dissertations were examined through document analysis to identify theses and dissertations relevant to research questions of the study. Following that, the documents were further investigated through content analysis. Sentences, paragraphs and sections are defined as coding units. Considering research questions of the study, a coding frame was developed and data were coded according to that frame.

Reliability
Research themes, research designs and research models, theories, concepts and frameworks that are explicitly reported in the theses and dissertations were included in this study. For a limited number of issues with regard to the theoretical background of the theses and dissertations, the first author interpreted and coded research findings into appropriate preset categories and the second author repeated the same process for reliability reasons. Cohen (1960) proposed that measurement of the agreement between two raters is calculated using the Kappa statistic. Accordingly, κ = .9329 SE = 0.0656 95% CI = 0.8044 to 1.061. According to Landis & Koch (1977), 0.81 to 1.00 refer to almost perfect agreement and Altman (1990) refers 0.81 to 1.00 intervals very good. Thus, it is thought that research findings can be considered as acceptable with a 0.9329 Cohen's Kappa value.

Findings and Discussion
Quantitative Information: Of all the studies accessed, 45% is theses and 55% is dissertations (Figure 3). A similar pattern can be seen when looking at the frequency of the published theses and dissertations (Figure 4) (2014) who reported that by 2013, MOOC numbers boom. This finding emphasizes that the incubation period of the MOOCs is over and that MOOC focused academic research is increasing rapidly.
Research Themes: When the theses and dissertations were examined according to the disciplines they belong to, it is observed that 51% of the studies are related to education, 19% comes from engineering and computer science, 12% refers to information and communication sciences, 8% is related to business and economics, 6% links to social science, and only 4% to media disciplines ( Figure 5). A detailed list of themes and disciplines is given in Appendix 2.

Figure 5: Research themes of theses and dissertations
MOOC research trends reveal three separate research strands that align with findings from other researchers. Veletsianos and Shepherdson (2015), Liyanagunawardena et al. (2013) and Gasevic et al. (2014) reported that the MOOC research was primarily conducted in Education, Information Technologies and Computer Science disciplines. According to Veletsianos and Shepherdson (2015) multidisciplinary, interdisciplinary, and cross disciplinary research represent promising venues for studying digital learning. On the other hand, MOOC research is currently dominated by the discipline of education (Gasevic et al., 2014) which risks to limit MOOC research. On this ground, we state that there is an important need: MOOC research from other disciplines is needed to provide a sound research base and to create an in-depth understanding of MOOCs. Several research fields are also left underexplored. For instance, MOOCers constitute an online learning community (de Waard et al., 2011) and research from sociology discipline would contribute to MOOC research.
Research methodologies, designs and models: When theses and dissertations were examined in terms of research methodology, it is seen that of all the theses and dissertations, 49% is qualitative, 21% is quantitative, 18% is mixed, 8% review and 4% is other (not specific) research methodologies ( Figure 6).

Figure 6: The most used research designs in theses and dissertations
Findings regarding research methodologies used in MOOC theses and dissertations have a different pattern when compared to previous MOOC findings. While theses and dissertations used qualitative (49%), quantitative (21%), mixed (18%), review (8%) and other (4%) research methodologies (Table 1), Gasevic et al. (2014) reported that of all the research submissions to MOOC Research Initiative (N = 78), 42% was mixed, 33% was quantitative and 24% was qualitative. Raffaghelli et al. (2015) also reported that of all the articles they examined, 20% was mixed, 15% was quantitative and 12% was qualitative studies, 30% was theoretical-conceptual, 15% was not clear (in research design) and 8% was design based research. Looking at the research model/design, the following frequencies occur (N = 51): case study is the most used research model (31%), followed by the correlational research model (14%), literature review (8%), explanatory design (6%), exploratory design (6%) and convergent parallel design (6%).
It is also salient that in addition to traditional data collecting tools such as questionnaire, interview and observation; new network based data collection tools such as social network analysis, user logs, internet and traffic ranks have started to be used in theses and dissertations. Considering the networked spaces as an ecosystem, it is promising to employ new data collection tools to be able to understand the structure of the networks and user behaviors on networked learning environments.
Theories, concepts, frameworks and models: There is a pattern in terms of employed theories, concepts, frameworks and models (lenses). Interestingly, 25 documents out of 51 didn't employ any theoretical lens to explain MOOCs, which equals to 49% of all theses and dissertations. Cumulatively, this value constitutes 37% of lenses (N = 68). Most of the lenses listed in Table 2 are distance education or technology related theories and they generally have a focus on online learning communities and distance education learners. It is also surprising that, though fewer, some lenses come from business and economics discipline. These findings (Table 2) demonstrate a similarity to Gasevic et al.'s (2014) findings in terms of employed theories, concepts, frameworks and models. In our sample however, the focal points of related theories, concepts, frameworks and models were online learning communities, social learning, and distance education learner characteristics. These findings reveal that (1) MOOCs have strong ties with the discipline of education, which explains why so many researchers from the educational discipline try to understand MOOC phenomenon and (2) other disciplines still perceive MOOCs as an emerging, popular research trend which result in research without theoretical underpinnings.
Further remarks: Researchers of this paper observed that MOOC theses and dissertations between 2008 and 2015 period generally examined xMOOCs which is a contrast pattern when compared to Liyanagunawardena et al. (2013) who reported that most of the articles in their sample covered cMOOCs. It seems that majority of research initiatives is shifting towards xMOOCs, which might be due to the popularity of xMOOC platforms. This coincides with what Ebben and Murphy (2013)

Conclusion and Future Directions
This research analyzed MOOC research trends distilled from theses and dissertations published between 2008-2015. Based on MOOC key events within the Gartner Hype Cycle, MOOCs are at the verge of Plateau of Productivity which means that there will increasingly be a diversity in MOOC applications in the future. Confirming this notion, some recent developments such as improved ID verification (recognizing typing patterns and submitting a valid ID with photo), diversity of types of the MOOCs (scheduled or self-paced), efforts to improve recognizability and accreditation (official verifiable certificates and specializations with signature track feature), increasing number of joint MOOC initiatives and partnerships among the MOOC platforms and universities provide some insights about the future of the MOOCs and its place in Hype Cycle. This study found that MOOC research is generally dominated by education, engineering and computer science, and information and communication technology related disciplines. Qualitative methods are currently preferred, but there is an increasing interest for quantitative and mixed research design studies. As a natural consequence of MOOCs as an education dominated research area, theoretical grounds are usually distance education related. Remarkably though, almost half of the theses and dissertations didn't use any theoretical framework.
This research serves as a complementary research by providing empirical research findings collected and discussed from a grey literature of MOOC studies, namely theses and dissertations. It is our hope that future MOOC research will focus on neglected research areas. Considering the findings of this research, the following implications may direct future research: • Current research trends focus mainly on MOOC learners and MOOC systems within an educational perspective. Thus, there is a need to conduct research within different disciplines to increase the diversity of MOOC related research findings. • Though in an increasing momentum-especially in 2014 and 2015-the number of mixed methods design is relatively low. More mixed method research might however eliminate possible weaknesses of qualitative and quantitative research design, and provide a more complete and comprehensive understanding of the MOOC phenomenon. • Though at present there isn't enough research to conduct a meta-analysis or meta-synthesis based study, such research would contribute significantly to MOOC literature.

Disciplines of theses and dissertations.
Theses and dissertations were presented for a degree in: Discipline