Icelandic Online: twenty years of development, evaluation, and expansion of an LMOOC

. In this article, the developers of seven Language Massive Open Online Courses (LMOOCs), Icelandic Online (IOL, https://icelandiconline.com/), describe the technological and pedagogical principles that have contributed to the program’s longevity. Development began in 2001 with a courseware system later upgraded to a multiplatform app. Over 80,000 users have completed one or more of the curated and pedagogically driven courses, which are monitored by a tracking system. The tracking system and follow-up surveys generate unique, large-scale empirical data enabling sustained engagement with participants’ views and behaviors as they go through the courses, some of which are offered in three different delivery modes. Finally, further development projects based on the versatile IOL non-language specific, multiplatform system are presented, including Virtual Reality (VR) projects, courses in other languages, and L2 literacy courses for children.


Introduction
In his review of research into Computer Assisted Language Learning (CALL), Gillespie (2020) identifies commonalities between empirical studies of CALL. He found that most studies involve few subjects, cover a short period of time, and are seldom followed up. This article offers an overview of a large-scale, long-term project (IOL), which involves tens of thousands of participants. The development of IOL started in 2001, and the first courses were launched in 2004. IOL offers seven free and open online courses to adults learning Icelandic as an L2. The courses have five proficiency levels from A1 to C1 on the Common European Framework of Reference (Council of Europe, 2001), and some are offered in three different delivery modes (blended, distance, and open self-directed). Over 80,000 users have completed one or more of the courses, and the program has received several national awards. It has an integrated tracking system to monitor students' behavior from the time they enter a course until they leave. Insights offered by the tracking system provide a unique opportunity to examine longitudinally the development and usage of this large-scale CALL program through sustained engagement (Arnbjörnsdóttir, 2004;Friðriksdóttir, 2018Friðriksdóttir, , 2019Friðriksdóttir & Arnbjörnsdóttir, 2018). The first part of this overview article describes the pedagogical principles that guide the development of the curated online courses, as well as the program's evolution from a local PC courseware platform (heavily dependent on Flash) to a multiplatform, multipurpose LMOOC application that is still relevant. The second part focuses on findings based on retention data generated by the tracking system and a follow-up survey of students' views on and experiences with the LMOOC, including different modes of delivery. The third part of the article outlines the ongoing multiple uses offered by the courseware program, including courses at primary and secondary levels in different languages and VR projects. The article ends with a few concluding remarks.

IOL: 20 years of development
The development of IOL began in 2001 in response to a call for online resources to support Icelandic instruction at various universities throughout the world and to provide courses for the many -but scattered -individuals worldwide interested in Icelandic language and culture. The initial seven courses were based on an authoring tool and course editor developed for this purpose that included over 40 patterns (including videos, audio, exercises, and Flash) that could be curated to create any course, for any language, at any level, and for any target group, without the assistance of a programmer.
The course development was pedagogically driven, taking into consideration that Icelandic is a lesser-taught language heavily dependent on nominal morphology in the initial stages of acquisition, which most adult learners find challenging. The technology serves the pedagogical goals. The content of the courses is Icelandic language and culture. The initial target group was adults at university, and a pedagogical framework was thus developed based on the needs and interests of that specific target group (Arnbjörnsdóttir, 2004). Later, a curated course for immigrants to Iceland was added using the same courseware, but driven by a different pedagogy with different content. IOL is aided by auxiliary features, such as dictionaries and grammar support from a Resource Center.
The use of technology to advance pedagogical aims accounts for the longevity of the IOL system. This includes a defined target group, clear SLA-based pedagogical principles, and curated, meaningful content that provides cultural context for the language. Participants work toward clear goals for vocabulary development, accuracy, and pragmatics, always in a meaningful context. The initial pedagogy was based on Chapelle's (1998) 'relevant SLA principles'. The material and the instruction is scaffolded, follows the principles of Focus on Form (Doughty & Williams, 1998), and is guided by the notion that frequency of input depends on whether morphological forms are rule-based or irregular (Pinker & Prince, 1988). Ten years later, the courseware system was updated to a multiplatform system that included both PCs and mobile devices, and now an app. An integrated tracking system and user surveys provide a good overview of the factors that affect student response and retention in the courses. The results are described in the next section.

Student behavior: tracking, retention and feedback
Studies showing low completion rates in LMOOCs have raised concerns about whether such courses are suitable learning environments for L2 learners, including that LMOOCs lack engaging forms of pedagogy and design strategies (Colpaert, 2014). Based on IOL's tracking system, a mixed-method study was carried out to investigate critical factors of retention and engagement in LMOOCs in the context of the IOL program. The study examined the tracked retention data (n=43,000) in all courses and delivery modes, as well as patterns of engagement and attrition across the courses. Second, a follow-up survey study (n=400) examined learners' views on the importance of the instructional design, tutor support, and other factors for their motivation, and the impact on retention as measured against the same learners' tracked retention data. Finally, the study investigated learners' reflections, elicited through the survey, on why they completed a course (112 informants) or disengaged earlier (62 informants).
The findings revealed, first, low completion rates in all courses, spanning 2.5% to 18%, depending on the course/mode. Course completion means learners who cover 100% of a course's content. The blended mode was most effective in retaining students, with a 14% completion rate, compared to 5% in the distance mode, and 4% in the open self-directed mode. The application of learning analytics found a regular pattern of attrition among non-completers across the three delivery modes, shown in Figure 1, with sharp attrition initially and relatively high attrition peaks at certain junctures throughout a course, some of which appeared very late in the courses. A pattern of user engagement was also found across the seven IOL courses, suggesting that many learners who would generally be considered as noncompleters of a course did, however, complete most of the content of a course. These findings (Friðriksdóttir, 2018) called for a re-evaluation of earlier methods of measuring course completion in the follow-up survey study, where course completion was defined as learners who had completed 80% to 100% of a course's content (Friðriksdóttir, 2019, in press).

Figure 1. Progress and parameters set for course completers and non-completers in IOL 2
Second, in the follow-up survey study, users in IOL 2 identified specific factors related to the content of the course and the tutorial modes that affected their engagement in and retention of the program, including gradual and scaffolded presentation of input (Friðriksdóttir, 2019, in press). The qualitative data analysis shows that multiple course-related and motivational factors affect learners' completion, such as satisfying course content (Friðriksdóttir, 2019), while external factors, such as time constraints, mainly explain non-completion of a course (Friðriksdóttir, in press).

The potential of IOL in other projects
Throughout the 20 years of the project, the IOL LMOOCs' infrastructure has played an important role in the development and support of other online projects.
This includes online courses for learning Finland Swedish 4 and Faroese 5 , and the development of IOL to support literacy in Icelandic as L2 for children. Another IOL-supported project was The Icelandic Language and Culture Training in Virtual Reykjavik which included features from artificial intelligence enabling learners to practice spoken interaction through conversation with animated characters. The Virtual Reykjavik project provides an interim learning space enabling L2 learners of Icelandic to play a serious VR game in order to develop communicative skills in a context-specific situation (Bédi et al., 2016;Bédi, Arnbjörnsdóttir, & Vilhjálmsson, 2017).

Discussion and conclusion
The IOL project has grown in recognition, and its development has been supported by university, Icelandic, Nordic, and European funding. However, securing operational funding remains a challenge, and it remains driven by individual initiatives. Academic recognition for the development work has been slow. Its remarkable longevity and relevance after 20 years may be credited to a principle that technology should serve the pedagogy. As Garrett (2009) suggests, "CALL" is not shorthand for "the use of technology" (p. 719). The content of each IOL course is curated to meet the needs of the target learners. Tracking student progress and engagement has shown that a range of significant determinants affect retention, including instructional design, course content, modes of delivery, and external factors. The limits of the current IOL system are its lack of effective conversation practice, although VR applications are promising, and the fact that learning objects do not provide adequate writing feedback. Both features depend on the appropriate language technology tools being available for Icelandic, which they are not. Future work includes the role of language and culture in student motivation and engagement in CALL (Gillespie, 2020). The new system has the added advantage that it enables graphs and visualizations for research to be performed by non-data specialists. This research is ongoing. Based on the available data gathered through IOL's tracker and users' surveys, one of the key takeaways from the IOL program is the value of an in-built tracking system for CALL systems to provide better understanding of student engagement in relation to their learning context. Also, data analysis highlights the importance of using instructional resources in CALL and SLA pedagogical approaches for the LMOOC learner.