Participation in the Virtual Environment of Blended College Courses: An Activity Study of Student Performance

This paper describes a study of success factors in the introductory semester of liberal studies blended courses offered at the bachelor of science level. The influence of student participation in the online course environment was examined, as measured by the number of times students logged into the learning management system (LMS) and average session length. These measures were correlated with final course grades to increase understanding of the participation patterns of successful students. The resulting patterns and their implications are identified. We observe that students with an intermediate number of logins and average session length tended to exhibit the optimal level of course performance with students who logged in near the low or high amount of times tending to receive lower grades.


Introduction
In an effort to provide students of a women's college in the Middle East with the highest quality learning experiences, flexible access to courses, and most accessible channels of communication among students and faculty, staff examined the potential for expanded use of open source software and platforms. The intent of the activity was to identify a rigorous and extensible system for teachers to build and share dynamic learning objects with students and between faculty. The pool of candidates for possible open Course or Learning Management Systems (LMSs) included Sakai and Moodle. Although these systems provided similar options, the college choose to pilot Sakai in an attempt to determine feasibility of use for blended courses, with the possibility of disseminating content and participation to other colleges in the region. Sakai has a modified approach to the traditional LMS, promoting the tool as a sharing or collaborative environment, hence the label of Collaborative Learning Environment (CLE). Sakai offers the core functions of prior LMSs with the addition of being open, which some interpret to mean free. Although there is no licensing for the software, the cost is in the human and technical resources required by hosting, maintenance, and programming. The college chose to outsource these responsibilities to a reputable hosting company in Europe, due to their proximity of time zone. The fee to host a two semester project for the college that enrolled 2,500 students on two campuses was $10,000 USD, which included all services as well as sufficient template modification to offer a branded site for the college. Both parties were aware that the cost was reduced as the intent was to produce proof of concept data, to enable full integration in subsequent years within the college, and potentially to expand use of the CLE throughout the multi-campus system.
The CLE hosting business's leaders and college staff worked via campus meeting and Web meetings, led by the host Project Manager and the college Technical Services Department Supervisor. The host provided training to the college technical staff, who were already experienced in managing LMS platforms. The goal of the pilot project was to create an effective and flexible system for student engagement, as well as instructor sharing of their resources, lessons, assessments, and so on. The anticipated outcome was that this system would be as good as, if not better than, the current LMS used in the college, which required a substantial licensing fee. The approach adopted for the pilot project was a quasi-experimental design in which both LMS platforms would be used in parallel for sufficient numbers of sections, instructors, and students to generalize the findings into other courses and campuses. Ultimately, the college found that Sakai could offer more than the commercial LMS, at a significantly reduced cost, and with no usability issues for faculty or students. This paper describes key findings from the open CLE pilot program. This study focused on success factors in the introductory semester of liberal studies blended courses offered at the bachelor of science level. The influence of student participation in the online course environment was examined, as measured by the number of times students logged into the learning management system (LMS) and average session length. These measures were correlated with final course grades in order to develop recommendations for blended course design and instruction, which must apply evidence for how students learn to the micro and macro levels of content chunking (Keeler, 2015). This study addressed the blended learning success factor of iterative and recursive design (Ferdig, Cavanaugh, & Freidhoff, 2015), and provides a model for programs that seek to refine their blended course design approach.

Blended Learning for Liberal Studies
Tertiary education is increasingly being redefined from site to service (Cavanaugh & Hargis, 2010) as the emphasis in education shifts from being a location where students have similar experiences Given as individual students reach mastery These shifts in the design and delivery of formal education have so far been largely made possible by technology, and their degree of priority has increased due to a range of economic and social conditions. Colleges are serving a wider range of students in degree programs, are under pressure to make efficient use of space and resources, are held accountable for each student's academic performance, are seen as key drivers in the nation's economic competitiveness, and thus, are depended upon to provide career-ready graduates. Because the technology now exists for differentiated instruction, these priorities are increasingly met through blended education programs that merge physical and virtual learning environments to serve students with greater flexibility and agility. This article describes a study of a semester-long college pilot program in which blended undergraduate courses were offered and data were analyzed to identify factors related to academic success in the courses. A blended approach to course design was especially appealing for the pre-professional students in the college who spent increasing time in professional settings during their baccalaureate degree programs, and could benefit from the added time flexibility afforded by blended course delivery.
In tertiary education, examples of technology-enabled blended education date from the beginning of educational technology. Classroom learning time has been enhanced, augmented, and partially replaced by technology for as long as the technology has been available (Ferdig & Kennedy, 2014).

266
The past decade has seen a steady increase in the development and implementation of blended education programs as technology has become more powerful and ubiquitous. However, it has been only in the past few years that deliberate blended education programs have developed (Clark & Barbour, 2015). This progress has been spurred in large part by open source learning management systems and more readily available education resources.
In  (Cavanaugh, 2009). Indeed, time was identified as the most significant factor in student success in 11 of 15 high enrollment blended courses (Liu & Cavanaugh, 2011). This finding encourages the design of courses to engage and motivate students to spend more time in the system. Specifically, Liu and Cavanaugh (2011) observed that the frequency with which students accessed online materials related to achievement in ways that seem to depend on the complexity of the course, in that more logins corresponded to higher achievement in introductory level courses, while fewer logins for longer sessions corresponded to higher achievement in advanced courses.
These findings guided the college open source blended course data analysis in spring of 2012.
The research question guiding this study is "Does student level of participation in the online learning environment correspond to student academic performance in a blended college course? If so, how?"

Setting and Participants
Data were collected during a full semester when the blended course design was introduced in a mid-sized four-year women's college in the United Arab Emirates. During this semester, 590 students and 14 instructors were registered in course sections created in Sakai CLE for courses that included classroom and online experiences. Seven courses were designed in-house for the CLE. Of these courses, one was developed and taught in a way that integrated physical and virtual interactions. This course (LSS 2533-see Table 2) is part of the bachelor-level liberal studies curriculum, and was taught by seven experienced faculty members for a total of 477 students using the same instructional materials, assignments, and assessments as the fully face-to-face versions of the course. Students were registered in LSS 2533 according to their program plans and schedules. Three hundred twenty-one students completed the course and were included in this study. Because the college serves only female National citizens of the country where it operates, students in the study sample represent a narrow ethnic and socioeconomic range in one large city.
Thus, demographic factors were not included as variables in the study. Because of the uniform backgrounds and entry levels of the students, differences among them in course performance may depend on their engagement in the course. Table 2 summarizes course information for students included in this study.

Instrument
The success of students in blended learning courses can be measured by the grades students earn in the courses and academic performance on standardized exams (Ronsisvalle & Watkins, 2005).
In the present study, student final scores in the course were determined by performance in a range of course assessments developed by expert panels for use across the national college system for these courses. Assessment methods were reflective of an undergraduate survey course, and included discussion, essays, limited-response exams, and projects. The assessments have been validated for their use over time and at multiple campuses. Student final course grade was used as 268 the measurement for success. Data logs generated by the Sakai CLE during the course included the number and length of student logins to the online course components.

Data Collection
The number of logins by each participant (a potential indicator of course presence during the weeks that the LMS was used) and average session length (a potential indicator of course engagement) were used as independent variables in our study. Login and final grade summary statistics are shown in Table 3 for the full 18-week semester of the course. Notice that while there was a wide range of login attempts (1-80), the interquartile range was much smaller with 50% of all students logging in between 10 and 24 times with a median of 15 logins. Typical session lengths lasted about 1-1.5 hours. A positive correlation was found between number of logins and session length (r = 0.19, p = 0.0006; Figure 1).

Data Analysis
We looked at two nested models for predicting final course grade based on using number of logins (Logins) and average session length (ASL) as predictors. In Model 1, final course grade is linearly related to both predictive variables. In Model 2, we define the quadratic relationship: Second order equations in the predictive variables were used to detect optimal performance at intermediate levels; for example, to allow us to determine if an intermediate number of logins predicts better performance than either too few or too many logins. All models were fit using the open-source software R. Due to the presence of extreme outliers (which could not be excluded on basis of incomplete information) and lack of normality in the distribution of residuals, the robust and resistant method of MM-Estimators, introduced by Yohai (1987) and implemented in R's MASS package (Venables & Ripley, 2002), was used to obtain estimates and standard errors for linear model coefficients. Significance of parameters was determined using asymptotic t tests.
Estimates for the optimal logins and ASL were obtained via standard formulas for the vertex of a paraboloid. The collinearity between logins and ASL was addressed by checking the variance inflation factors (VIF) associated with each term in Model 1.

Results
Model 2 significantly outperformed Model 1 demonstrating a parabolic relationship between our response variable (course grade) and both predictors (logins and ASL) as shown in Table 4. Logins appears to be a more significant predictor than ASL although since these variables are collinear, we leave both terms in the model. Both quadratic coefficients in Model 2 are negative implying that course grade is predicted to be highest at intermediate values of both predictors, as shown in Figure 2. We approximate that the optimal combination of login times and session length was approximately 22 logins lasting around 63 minutes on average.  (Cavanaugh, 2009;Liu & Cavanaugh, 2011). Since number of logins is positively correlated with ASL (Figure 1), this would also explain the dependence of course grade on number of logins. Alternatively, we conjecture that overly frequent logins may indicate a distracted student. Such a student may be logging into Sakai CLE frequently, but spending the bulk of their login time switching between the course Web site and non-course related Web browsing. As long as the student continued to stay logged in to Sakai, a long session length would have been recorded even though the student did not spend much of their login time on course related exercises. Such distracted sessions may, in turn, beget more distracted sessions, leading to a further increase in the frequency of logins. Finally, a third explanation is the classic correlation does not imply causation paradigm. It is possible that students who logged in too many times were less prepared students and the additional login time was not sufficient to overcome these initial aptitude gaps. Further study accounting for entry level skills is needed to parse this possibility.

Conclusions and Implications
We examined student participation in the online components of a blended undergraduate college course in order to identify the extent to which the number and length of logins corresponded with student grades in the course.
It was not the intent to compare the effects of learning in blended courses with other learning environments or to evaluate the specific design approach of the course. This study focused on student engagement in the online components of the course in order to guide students, instructors, and course designers. Each blended learning program is unique and designed for a specific purpose (Ferdig, Cavanaugh, & Freidhoff, 2012). This blended course was intended primarily to expand the learning time for native Arabic-speaking students who used the online tools and resources at the times of day and for the length of time they needed to enhance their learning, given the wide range of logins and session lengths recorded. For these students, who also met in class weekly, one to two logins per week of just over one hour in duration each related to the highest course grades. This result confirms the claim that variation theory underlies effective blended learning experiences because a range of activities maintains student engagement (Oliver & Trigwell, 2005). In keeping with variation theory and this study's findings, effective blended learning courses can be designed to alternate physical and virtual experiences that require an hour of online time.
Education in any learning environment is a complex, challenging professional practice; education across various learning environments multiplies the complexity and challenge. Formal research is needed in order to illuminate promising practices for each intersecting student and learning goal.
This paper sheds light on one example. Data from practice must be coupled with data from