Universities Confront ‘Tech Disruption’: Perceptions of Student Engagement Online Using Two Learning Management Systems

do, and describe their similarities and differences. Then we report findings from a survey and focus groups with students who used both LMSs in these courses. Here we examine students’ subjective perceptions of Classroom and Blackboard, and account for how teachers and students engaged with each other through each LMS. We also discuss our own experiences with student engagement through different LMS throughout our teaching careers. Finally, we conclude with a discussion of how claims about disrupting educational practice through technology compare to the actual experiences of teachers and students who use these systems.


HOW STUDENTS LEARN ONLINE
Research on teaching and learning online dates from the mid-1990's to present-day. Student preferences for learning online, what they actually learn in this environment, and the online structures that support interaction are often the focus of research. Reviewing this literature, Tallent-Runnels et al. (2006) note that students view online instruction positively for its convenience and autonomy, but this view is more likely to be held by learners with proficient computer skills. Research on online learning has also found that students prefer courses with a high degree of interactivity with the instructor guiding student participation (Keefe, 2003). Student comfort and expectations for learning online are also factors that shape their perceptions, and to some extent, their outcomes. Faux & Black-Hughes (2000), for example, compared traditional, online, and hybrid sections of an undergraduate course in social work. Students showed the most improvement in the traditional course, and 41% of the students did not feel comfortable learning via the Internet in their online course. Students wanted more instructor feedback and auditory stimulation; they preferred listening to, rather than reading about, the material on their own (Tallent-Runnels et al., 2006).
Research also suggests that students shallowly participate in online discussions. Thomas (2002) examined undergraduate students' interactions in online discussion forums, concluding that students' cognitive engagement peaked at the point where they could identify correct information or discern differences in viewpoints. Constructivist learning theory suggests this type of interaction is insufficient to make knowledge construction possible, as students learn when their current view of knowledge is challenged, reformed, and synthesized through their interaction with others (Vygotsky, 1978;Tallent-Runnels et al., 2006). One possible explanation for this outcome is the structure of online learning environments, which are essentially featureless web pages that must be shaped and configured by the instructor. Direct guidance from the instructor (Christopher et al., 2004), improved instructional design (Berge, 1999), giving immediate feedback (Mikulecky, 1998), allowing students to get to know one another (Wilson & Whitelock, 1998), and forming small groups (Tallent-Runnels et al., 2006) are strategies that appear to create a deeper understanding of course content.

HOW LEARNING MANAGEMENT SYSTEMS AFFECT STUDENT LEARNING
As researchers began to understand how students learn online, the learning management system itself became an object of study. Research has identified several factors that influence user satisfaction or dissatisfaction with an LMS, and how LMS design can improve student engagement. In order to increase satisfaction with online learning, education researchers pinpoint several design "best practices." These include: 1) setting clear guidelines for interaction with students through the LMS, 2) facilitating meaningful cooperation among students, 3) allowing students to present course projects, 4) including classmate and instructor critique of these projects, 5) establishing trust between instructor and student by responding promptly to their concerns, and 6) communicating high expectations and praising quality work (Bradford et al., 2007). In addition, learning management systems, at a minimum, should "support active student engagement, meaningful connections between segments of the course, easy communication, and formative feedback on work" (Rubin et al., 2010, p. 82).
Other researchers have illustrated how instructors and students are more engaged and have a stronger teaching and cognitive presence within the LMS when it is intuitive and easy to use (Nicole & Mcfarlane-Dick, 2006;Rubin et al., 2010). For example, in Islam's (2014) study of Moodle, instructors were dissatisfied when the system was slow and unresponsive, too complex to use, and lacked visual appeal. Students were more likely to be dissatisfied when the system was slow and unresponsive, lacked reliability, and limited social interaction. Moreover, cumbersome and time-consuming navigation of web pages appears to frustrate instructors and students alike. Blackboard, for example, requires approximately "twelve distinct steps per student to provide feedback" (Rubin et al., 2010, p. 83). Therefore, the ease with which students and instructors can communicate and make connections through an LMS could have a profound impact on both the quality and quantity of feedback that instructors leave for students as well as the amount of time students spend reading feedback.
Lack of "presence" is another factor that can lead to dissatisfaction with learning management systems. Wegerif (1998) quotes a student in his study of asynchronous LMS as stating, It is a cold medium. Unlike face to face communication you get no instant feedback. You don't know how people responded to your comments; they just go out into silence. This feels isolating and unnerving. It is not warm and supportive. (p. 38) Though Wegerif's study took place close to 20 years ago, the critique still stands. Traditional LMS environments offer a static discussion board where threads can be difficult to access, students are faceless user names, and interaction is minimal.
Wegerif's description of an LMS is common and emblematic of one major concern within education scholarship: how online environments can enhance or hinder student engagement. "Student engagement" is broadly defined as a phenomenon in which students take charge of their learning and create their own knowledge. It represents an array of psychological, cultural, and behavioral practices whereby students interact with their studies and institutions for personal growth and fulfillment (Coates, 2006;Kahu, 2013). Alienation, a concept advanced by sociologists, explains what student engagement is not--that is, separation from something outside oneself (Geyer, 2001). Subsequently, engagement includes several dimensions, such as student motivation, interaction with teachers and classmates, institutional and non-institutional support, and active citizenship (Zepke et al., 2010). Student engagement is not solely an individualistic trait, and many societal changes have shaped the way it is presented, from the commodification of education (Smith et al., 2007, p. 684) to cultural shifts in the goal of university study (Krause, 2005).
Currently, studies privilege the technical and managerial aspects of learning where students are typically treated as "users" rather than "learners," and how students use an LMS to engage with the course material is seldom examined (Coates, 2006). Moreover, how the structure of an LMS affects student engagement is not well understood. Student engagement across different LMSs may unfold differently across systems. Because the number of LMSs have multiplied in recent years, comparisons of different LMSs and student engagement needs further investigation. For our purposes, student engagement involves personalized exchanges whereby students interact with their instructors and classmates as a "whole person." Therefore, the goal of this study is to understand how student engagement, as both a process and an outcome, is subjectively experienced as students adapt to a new LMS (Classroom) while simultaneously using a familiar system (Blackboard). In the next section, we describe the specific features of these systems to set the stage for our findings.

WHAT LEARNING MANAGEMENT SYSTEMS DO
Even though the Internet developed in an academic setting, its adaptation as an instructional resource has been characterized as relatively slow and inefficient (Jungwirth & Bruce, 2002). The advent of learning management systems was seen as a way of successfully integrating electronic learning into the traditional brickand-mortar classroom, as these systems allow instructors to organize learning materials, activities, and assessments (Hall, 2002). Falvo & Johnson (2007) suggest that an LMS is best used as a means through which students and instructors can discuss course content outside of the constraints of time and distance. As of 2009, a report by the American Society for Training and Development revealed that 91% of their respondents utilized LMS, with the percentage only increasing since that study (Chung et al., 2013). Google Classroom, a relatively new LMS, and Blackboard, a system that is 20 years old, are under investigation in this paper. They offer similar features that are delivered in different ways.

Google Classroom
Google Classroom launched in 2014, as a part of the G Suite for Education, which features Gmail, Calendar, Contacts, Docs, Sheets, Slides, Drive, and Hangouts (Singer, 2017). Typically, an information technology (IT) administrator enables the G Suite for Education suite for instructor use on a university domain. However, with the latest update (as of March 2017), an IT administrator can enable use of G suite from any personal Google account. This means that Classroom is open to anyone, including those who may not have a university email domain. In G Suite, individuals are either assigned the role of teacher or student. Instructors create classes, which are housed within a separate folder in the respective user's Google Drive. Student enrollment is not automatically populated in Classroom; instead, instructors invite students to the course by emailing them a class permission code. Instructors have the ability to post announcements, assignments, quizzes, and any other information on the Classroom "stream." The stream functions like a Facebook feed, with "posts" that are read in descending order. Students can also comment on posts, similar to comments on a Facebook feed. Instructors have the ability to attach YouTube videos, links, and files (which are uploaded to Drive) to any post (Google for education, n.d.).
Instructors can also assign due dates for assignments or schedule an assignment to be posted at a later date. Instructors can also create a template of the assignment for each student, which is added to their own individual Drive account. This means that students can open the assignment (using Google Docs for word processing), and work within their own document designed by the instructor. Furthermore, each assignment has its own page for grading, where student names are listed next to their submission. Instructors and students can interact on each assignment in three ways: 1) through "private comments," which are only visible to the individual student and their instructor; 2) through comments directly on a document (as long as it is in a Docs, Sheets, or Slides format), and 3) through a synchronous chat window. What is particularly useful about this feature is that students can work on the same file together, and see the changes made in real time. Google calls this the "Real Time API" and it is what distinguishes Google from other LMS offerings (Google for education, n.d.). There are no licensing costs associated with Google Classroom. Google Apps for Education primarily makes a profit from installation fees and hardware such as Chromebooks, monitors, and projectors (Singer, 2017).

Blackboard
Blackboard Learn is an LMS developed by Blackboard Inc. in 1997. In 2015, Blackboard Inc. released an upgrade, called "Ultra," but our institutions have not yet chosen to use this new version. Students and courses are automatically added to Blackboard. An instructor has the ability to build a course using separate pages which are collated into links on a side menu bar. Instructors can build discussion forums, assign groups, send announcements, create assignments and assessments, and grade completed work . The plagiarism-prevention service, Turnitin, can be integrated within Blackboard to automatically screen student work for unoriginal content. Work can be instantly graded using Blackboard's Test Manager function for quizzes and exams. Discussions are asynchronous, and messages appear as an outline or threaded "forum." Blackboard also tracks student usage of courses and submitted assignments, which allows the instructor to monitor late assignments and student progress. Instructors grade student work in a format similar to a spreadsheet, and students view all of their assigned grades in one place. Blackboard also includes a calendar for each course, and current due dates for assignments are displayed in the "Welcome" area when students log-in (Blackboard learn: LMS feature showcase, n.d.). Blackboard rents subscription licenses to universities for a starting fee of $50,000 per year, but as universities add more content, the price can increase up to $160,000 per year (Olsen, 2001;Feldstein, 2006).
In sum, technology has been developed to "disrupt" traditional classrooms, but students are frequently treated as consumers of an LMS rather than active participants in their learning. Given the proliferation of both hybrid and exclusively online college courses, it is important to understand how these online systems affect the way students engage with each other and the material.

METHOD
This is primarily a descriptive, qualitative study of student experiences using both Blackboard and Classroom. Our sample consisted of students in our upper-division sociology courses (Research Methods, Classical Theory). Our university's Institutional Review Board approved this study in 2015, with renewal each year until the data analysis was completed. We conducted our research at a regional, four-year university in the San Joaquin Valley region of California. The geographic location is significant as six of the counties within the San Joaquin Valley have the highest percentage of residents living below the poverty line in California and are listed among the most impoverished in the United States (Burd-Sharps & Lewis, 2011). Forty-five percent of students at our target university are Hispanic; eightytwo percent are first-generation students; and sixty percent of students at the university receive Pell Grants (The College Board, 2013). Students were offered no incentives for participation except their own desire to help with this project. When we announced this study to students, we clarified that their participation in the study was not related to their performance in the course. Students had the option to participate in both a survey and a focus group.

Survey
The survey was administered online using Qualtrics. In the Spring and Fall of 2015, we invited 153 students to participate in our survey about their technology use on and off campus, in addition to their satisfaction and perceptions of course management systems. Seventy-two students responded (n=72) for a response rate of 47%. The sample for this survey was derived from our own sociology courses (3 sections of Classical Theory, 1 section of Contemporary Theory, and 1 section of Research Methods).

Focus Groups
Four focus groups were conducted with students in 2015, either one hour preceding or following the regularly scheduled course time. Each focus group ranged between 8 and 20 students, with a total of 54 students participating. The instructors provided pizza and soft drinks to show appreciation for their time. Students read and signed a consent form before the focus group began. We informed them that the focus groups would be audio recorded and transcribed, and that their participation would remain confidential (identifiers removed in transcription).
Questions probed students to consider their experiences using both LMSs, and how each affected their engagement with peers, instructors, and the material. Sample questions are as follows: "Which LMS helped you understand the course material better?" "How have your feelings regarding the use of each LMS changed throughout this semester?" and "What was it like using each LMS to submit homework, communicate with other students, communicate with the professor?" Other questions that emerged organically from the focus groups include: "What helps you succeed in your online courses?" and "What helps you to be comfortable interacting with your classmates on Classroom compared to Blackboard?" The focus group sessions were transcribed and thematically coded line-by-line. Codes were then organized according to the recurring themes of usability, interaction, and facilitation of learning.

RESULTS
The study was led by two assumptions, 1) that the process of adaptation to a new course management system is worth examining and 2) that the features and functionality of an LMS can impact student engagement. We begin by reporting focus group findings, and at the end of this section, we describe our history with various LMSs over the years, our experiences using Google Classroom and Blackboard, and our impressions of student engagement under each LMS. Table 1 summarizes what students liked and disliked about each LMS. Table 2 reports findings from our survey about LMS platforms used at any university and student satisfaction with each LMS. In general, students appreciated the streamlined appearance of Classroom, the ease with which they were able to collaborate on group work through Google's real-time API, and the flexibility and utility of Classroom's mobile app. Blackboard's gradebook was preferred over the presentation of grades through Classroom; students were also comfortable with Blackboard as a familiar, albeit problematic, technology.

Student Experiences
We began the focus group by asking students what platform they primarily use to access Classroom. This question was important to understand because each platform offers a slightly different experience that can affect how students engage with the course. Table 3 indicates that a majority use Classroom on a combination of phones, tablets, laptops/PC's. To gauge how many students experience the integration of Classroom and its related applications (Docs, Sheets, Slides), we asked if students use the Suite offered by Google, or if they used a more traditional program like Microsoft Office, OpenOffice, or Pages. Across all the focus groups, half of students reported using the G Suite for their primary word processing, database, or presentation programs. Overall, three themes were identified from the focus groups: usability and learnability; interactivity and individuality; and facilitation of learning.
Usability and learnability. This theme encapsulates the experience, or usability, of a trouble-free interaction with software characterized by web pages that are simple to navigate and intuitively organized to help the user find what they are looking for with ease (Nielsen, 1994). This theme also incorporates learnability, which is a term used in e-learning research to refer to how users learn to use an LMS (Kakaseveski et al., 2008). We found that some students were hesitant to learn a new LMS, since Blackboard was familiar to them. One student described learning a new LMS as "scary." She had struggled to master Blackboard, and learning a new LMS "was kind of intimidating." She lamented, "it frightens me because I feel like I'm not up to par with everyone's skills, especially when I have been out of school for so long. That was what scared me, oh my gosh, I have to learn something new on top of the material." Several older re-entry students shared this sentiment. They remind us that learning an LMS and learning the course material are not separate tasks for students. In fact, a student may internalize their capacity for learning through comparisons to a generalized other whom they perceive as mastering the material (and online LMS) more quickly and efficiently.   Indeed, the situation for our students was unique in that some were using two LMSs at the same time. Of this situation, a student remarked "I thought it was going to be complicated...my first thought it was gonna be a hassle trying to logon to Blackboard, and now this." Another student stated that it took her about a month to feel comfortable enough to learn a new LMS, reminding us that adaptation is not instantaneous. Students suggested one way to remedy LMS anxiety was to offer a tutorial: "At the beginning of the year...maybe give us a little bit of time to go through it. If we have like an hour, have everybody walk through it. There might be some people that have never gone through Google Drive." Testing Classroom in a hybrid course had an important advantage in this regard. The fact that students met in person each week gave them opportunities to help each other learn the system. A student mentioned how a classmate helped her set up Classroom on her laptop and phone, which she greatly appreciated: "She was my lifesaver, because I wouldn't have known what to do. It was really hard in the beginning, but once you start playing with it, it just made sense." Student hesitation for learning a new LMS suggests that adapting to a new course management system is an emotional process that some students view as burdensome. While anxiety about learning a new system was a theme, many students in the focus group were surprised by Classroom's user-friendliness. Students reported that usability mattered to them because an LMS can make it easier or more difficult for them to access course materials and updates. For instance, one student compared the log-in process of Classroom and Blackboard across different hardware platforms that he uses: Google is way more user friendly, I mean it's so much easier and more accessible than Blackboard is. I have three different devices and Blackboard you have to log into the computer every time. You are able to work on a lot more stuff on Google.
Another student suggested that downloading files to her mobile device helped her access course materials without the cumbersome process of logging-in or finding a Wi-Fi connection. This allowed for easier access to electronic course readings: Once you upload the reading, I like that you don't have to have internet to read it. So if I just want to look at documents just for my reference, I don't have to go into Classroom and then go into scroll and back to where it was...I can just go into my documents app and [the documents] I need are right there.
In one focus group we discussed how our students, as parents, were using Classroom along with their children in the K-12 system. Pivoting from this discussion, a student concluded that Classroom's usability was not age-specific. She noted that Classroom "has the tools that are necessary for college or higher level and if it's simple enough for first graders to do, that's a really well written system." One aspect of Blackboard that irked students was the difficulty using the discussion board. A student remarked, "it's really hard to talk, or figure what conversation you're in on my phone." Using Blackboard's discussion board on her laptop is easier, but she cannot always carry her laptop with her because she is "always on the go." Students also seemed to prefer Classroom's structure for organizing assignments, as several students liked how Classroom clearly marked their work as "done" or "not done." Said one student, "Blackboard is so complicated and feel like Classroom is so much easier. I like that I can just look up assignments to see what I've done or haven't done." Another student liked that the Classroom mobile app sends out a notification when a new assignment is posted: "Instead of having to weekly check, instead it's like 'aww, I got a new assignment alright, there you go.' It's really helpful." The features on Classroom appeared to help students manage their submitted work, assignment due dates, and reading tasks. Overall, students overwhelmingly praised Classroom as more user-friendly, and with a shorter learning curve, than Blackboard.
Interactivity and individuality. Interactivity is a nebulous concept in higher education. People engage with university study in different ways, but our participants appeared to value this concept as a feature of online learning. We did not quantify instances of student engagement; instead, focus groups illustrate how student discourses express engagement as a social value. Students in the focus groups appreciated an online environment in which they could view individual instructor comments on papers, and could communicate more easily with their peers. As for the former, students reported feeling engaged when they could respond to instructor comments on papers directly in-line with the text. Said one student, I liked the feature in Classroom where you can comment because at least I can leave something, versus having to go to my email and emailing a professor. It's like more personal in Classroom, it allows you to talk to the professor more, one-on-one.
The professor's ability to respond to a student in a timely manner was also an important LMS feature. Students quickly learned that they would receive a comment much faster through Docs or Classroom than if they had emailed the professor or sent a message through Blackboard. One student appreciated this, stating, "you'll respond right away...rather than having to email and you have to think, 'OK I better check my email to see if I got a response.' With [Classroom] you can just see that I got a response right away. This is something that is easier to access than email." Email was not a completely outdated mode of communication, however, as many students appreciated that they received an email when a professor graded an assignment. These efforts seemed to make students feel like they were interacting with a real person instead of a faceless institution.
Student interactions with professors was an act we could confirm from our own direct experience, but how students interacted with each other through the new LMS was hidden from us. Unless students invite us to collaborate on a document, Google blocks instructors from viewing student chat and comments to each other in Google Docs. Therefore, we probed students: did Classroom change the way they worked together on assignments, as opposed to Blackboard? Did students communicate with one another through an LMS the same way they would in class? Students reported that Blackboard offered few avenues to make personal connections with their classmates. In the words of one student, "[Classroom is] a lot more personal as opposed to Blackboard...when you're in Blackboard it's just like 'oh, that's Classmate N'." Recounting how her group collaborated using Google Docs, one student said, "We used [the chat window] when we were working on a document...we can all do it together, it's so awesome. It's great to get ahold of classmates." Another student described how group work was accomplished: "We are all working on it at the same time. Saturday, Sunday, the whole group was in the program and we were asking questions." This particular point about tracking down busy classmates to accomplish tasks together was a reoccurring theme. It appears that any web application that can help students contact their group members is meaningful to them. Using the G Suite apps was easy, remarked one student, because "you can be at different places and each one from the group can work on [an assignment] at different times and it saves [automatically]." Moreover, working on documents at different times, while clarifying what needed to be done next, appeared to foster amicable teamwork with classmates. One student said, "my friend wasn't in class that night but she was able to see all of the comments I had put on there. So she knew what she needed to do in order to fix the errors or improve [the paper]." Students in our sample appeared to prioritize interactivity and individuality in their online learning environments, and they felt that Classroom did a better job than Blackboard of creating connections with professors and classmates.
Facilitation of learning. This study did not document a change in student mastery of the subject matter as a result of a particular LMS, but we did ask students to reflect on how each LMS facilitated their learning. Considering this question, students recognized that the bulk of the work of learning was on them, and that no technology could do the job for them. One student observed that an LMS "certainly gives us the tools to [learn], whether or not we do that is up to us." Despite this limitation, students frequently mentioned how an LMS can help direct their attention to the most important material or comments from professors. One student appreciated how Classroom encouraged students to engage with instructor comments because of the architecture of delivering feedback: I have gotten feedback on Blackboard for my class. You submit your paper and there is like an example of your cover sheet and it has the grade. Then, underneath it, it's got comments that he made on my paper, but it's not like in your face like Classroom where it's like BAM! There is no way around or missing [instructor comments] because it's highlighted and in color.
Irrespective of this student's enthusiasm for Classroom, it is important to note that seeing the comments and acting on them are two different things. What is the outcome of this feedback structure, and does it even matter how feedback is presented if students do not learn from it? Students offered little clarity on this question, and we still do not fully understand what they do with our feedback delivered through the LMS, or how they take it and apply it to new lessons. Only one student provided evidence for how the feedback in Classroom changed how she wrote papers: I never have had an assignment where I have gotten detailed feedback, where it was like change this wording to something else, for instance. Seeing [this feedback] helped me edit my papers before I turned them in.
Interestingly, for facilitating their learning, students recognized that an LMS is just as important as selecting an engaging textbook or developing an interesting lecture. When asked to explain what it means when a professor experiments with a new LMS, one student said he felt appreciated, ...because it almost seems like they value our time. It's like when you're looking for a book that is going to teach the class the best way, right? It's the same thing with the mobile app or with Google Classroom and Blackboard, you're trying to find the best tool for your students to learn. I feel like if they aren't taking the time to look for that, they aren't caring enough about our education enough to know that hey, accessibility is key for us to learn.
Students in our focus groups were notably appreciative of efforts to find an LMS that best facilitated their learning. For our students, Classroom appears to deliver more useful tools to help them learn if they wish to do so, but there is not enough evidence to conclude that Classroom or Blackboard enhanced their learning in measurable ways.

Instructor Experiences
As instructors, we collectively have experience with Blackboard, WebCT (now owned and operated by Blackboard), Moodle, and Google Classroom spanning a twelve-year period. As both teaching assistants and instructors of record early in our careers, we found LMS discussion forums housed their own "special genre" of student writing: students parroted others in an attempt to appear engaged. Compared to voluntary internet forums or face-to-face class discussions, students on an LMS discussion forum lacked authentic, spontaneous engagement. Typically, students only responded to one another with the minimum required word count, nothing more or less, and the depth of discussion rarely progressed beyond "I agree" or "I disagree." Part of this was likely due to the tree-and-stem design of WebCT/Blackboard discussion boards. This design forces students to actively "click" on each other's posts in order to read them. We believe that extra step interrupted the flow of "conversation" and appeared to lead to most students randomly selecting a post and responding to it, thereby creating the impression of inauthentic and forced communication. Heated arguments and voluntary comments to students were rare. In the mid 2000s, when Blackboard controlled the majority of the LMS market, we both found its interface to be glitchy and clunky. The LMS was frequently taken offline for maintenance, anywhere from two hours a day to two hours a week, which hindered our ability to work at times most convenient for us. The system's propensity for crashing mid-grading resulted in lost comments and grades. As a result, we often graded the same assignment multiple-times. Between our initial exposure to WebCT and Blackboard in the mid 2000s, and beginning this study in 2014, very few significant changes were made to the format or the functionality of the LMS.
Our IT administrator responded positively to our request to try a new LMS in the Fall of 2014, though he claimed that Classroom lacked many of the features of Blackboard (our university's dedicated LMS) and that we may find it minimal for our needs. When we made the switch to Google Classroom from Blackboard, only one other professor out of our university's approximately 400 active faculty members was using Classroom at the time.
We both found Classroom to be intuitively simple and streamlined which translated to an extremely short learning curve. Minimalism was clearly a feature, not a bug, of the user interface. In contrast to the warning from our IT administrator, neither of us missed Blackboard, preferring the ease with which we could accomplish the main tasks we want an LMS to do: uploading and accessing course readings and links; engaging students in discussion; and providing feedback on student assignments. An unexpected bonus that we experienced with our writingintensive classes was that we could comment directly on students' work through Google Docs and see how students dealt with our comments and critiques through Docs' historical track-changes function. Having students work through Google Docs also allowed us to track students' participation on group assignments. The ease with which we can now grade student work online has eliminated the need for collecting printed work, and students--plagued by printing costs--seem to appreciate this option. Accordingly, through Classroom, we believe students took greater control of the material and their learning process, using the Classroom platform to ask each other clarifying questions about assignments and readings, set up study groups, and more actively engage with us as the instructors.
Another useful feature is the ability to reuse posts from previous classes. This is similar to Blackboard's feature of "cloning" a course. However, by saving posts instead of the whole course, posts can be conveniently tailored and distributed with a new deadline. Student interactivity on the Stream page also felt more natural, perhaps because of the Facebook-like comment structure where students could reply individually (using an @username feature), with replies also publicly visible. Posts and grades are instantly saved in draft form, preventing the loss of data from a computer or browser crash. The interface is pleasing to the eye; Classroom looks and acts more like a mobile app than a website.
Classroom has several flaws. The "questions" feature, which allows instructors to post a multiple-choice question to monitor student comprehension, is cumbersome and does not work well in writing and discussion based courses. The gradebook also has limited functionality. Instructors cannot assign half points or use letter/text based grades, and Google offers no helpful reminder that grades need to be returned. Exporting grades to Excel or Sheets is easy, but there is no functionality to upload your own spreadsheet, which Blackboard does allow. Furthermore, there is no similar "course preview" feature as in Blackboard; we added each other as students to our courses so we could see how students view Classroom on their electronic devices.
Because the gradebook, assignment submission page, and course log-in look different for students, this obstacle created difficulties when offering a tutorial on Classroom at the beginning of the semester. Likewise, delivering tests and quizzes using Google Forms is simple enough, but Blackboard does have more options for test question formats. The use of Calendar, Hangout chat and video, and Google+ is only available if an IT administrator enables these applications (and ours did not). Thus, we cannot comment on the usefulness of these tools. That said, in our opinion, Classroom has many advantages over the traditional LMS. Perhaps this is why the number of professors using Classroom has increased on our campus, and even the writing center uses it to conduct student tutoring. In the future, we would like to launch a campus-wide survey of LMS use to generate comparisons of online instructional formats.

DISCUSSION
As a tool, an LMS can foster more intimate connections online between instructors and students. In-text comments and real-time collaboration can help students work together and reflect on their own process of writing and editing. Moreover, since the average 18-24 year old checks their phone approximately 82 times per day (Deloitte, 2015), mobile app notifications are an important part of this engagement process. Students were enthusiastic about the ability of mobile app notifications as an external motivational tool to stay committed to the course. In addition, the themes that emerged from the focus groups illustrate how students value an LMS that is usable, personable, and useful to their learning. Comparisons between two LMS, Blackboard and Classroom, revealed that the functionality of an LMS matters to the student experience. Not all LMS are equal and replacing one with another will not produce the same sense of satisfaction among students and instructors. Most notably, our analysis suggests that students more actively engage with a course when they are reminded that an instructor is evaluating their work and feel empowered that they can respond to these evaluations. Clearly, Google Classroom provided more opportunities for this process to unfold.
The primary limitation of this study is that the conclusions are based on perceptions of learning experiences. An objective assessment of student learning outcomes was not conducted, so we cannot speak to the observable differences between LMS in student performance. However, students' personal perceptions of their aptitudes are important in understanding their lived academic experiences. We also cannot speak to the functionality of different LMS in fully online classes, though the preliminary results from this study suggest that students would be more fully engaged with the learning process using an LMS with a more user-friendly interface such as Classroom.
Furthermore, students may have preferred Classroom because their Blackboard courses were taught in a more passive, less interactive way. However, this leads to the question of what causes the passivity in the first place--does the LMS itself depress student and instructor interaction, or is it mainly a factor of instructor quality? If the latter, is it up to the instructor to overcome these user barriers and create meaningful interactions in the online spaces given to them? This study did not determine if using one LMS, absent the instructor, inherently creates more student engagement over another LMS, but the qualitative data does suggest that LMS design can create conditions (like usability and navigability) that allows engagement to flow more naturally and easily between students and instructors. Our observations as instructors bolster this conclusion. Since we were more enthusiastic about logging into Classroom, we were more likely to give students feedback. We were less likely to do the same activity in our Blackboard courses because we struggled with Blackboard's user interface. At the same time, this inclination has the potential to bias our findings. Even if we had hosted multiple sections of our courses on different LMS, our preferences for an LMS became clear as the study evolved.
This research answers important questions about the "what, how, and why" of online learning. We reveal what students prioritize when using technology to learn outside the face-to-face classroom; how technology can make their role more efficient and gratifying; and why they fear new technologies. These findings are connected to wider literature on student barriers to learning (Blum, 2005), and how students use technology in the education setting (Hiltz & Turoff, 2002;Krentler & Willis-Flurry, 2005;Jackson et al., 2011). This study also relates to literature on the meta-cognitive processes that are used to plan, order, and assess learning (Pintrich, 2010). Learners build concept maps over time; they may not "get it" in the classroom, but later on after they have had a chance to reflect on what was presented (Chen, Liang, Lee, & Liao, 2011). This fact illustrates the importance of keeping students involved in learning when an instructor is not directly present. An LMS can do this well by reminding students that an instructor is reading their writing or that a student is helping with a shared project, therefore decreasing their alienation within an institution that is otherwise impersonal and non-responsive.
While an LMS can benefit university instruction, claims about how these tools are transforming education are disingenuous. We began this paper with the idea of disruption, i.e. the tech industry's purported goal to revolutionize a mode of education that has persisted since the Industrial Age. However, at its core, education in the Information Age is no different than a traditional, brick-and-mortar classroom. Teachers must still guide students through a disciplinary approach and offer helpful feedback, and students must discover new knowledge on their own. Rather than disrupting the status quo, an LMS modestly replicates traditional modes of knowledge transference (e.g. watching videos, getting people to talk to each other, or delivering feedback). Consequently, simply adding a technology to traditional teaching does not necessarily improve learning or engagement. In order for technology to be used effectively, an instructor must invest in activities to encourage interest in the subject and to furnish opportunities to learn in nontraditional ways through that technology (Rice, Cullen, & Davis, 2011). A human being must still create the conditions for learners to creatively explore ideas, seek out resources to help understand topics, set high performance standards, or push for an understanding of things that are puzzling (Coates, 2006).
That said, this goal may be difficult to achieve when faculty members lack instructional support on campus for online instruction. Research has found that instructors want more technical support (Frith & Kee, 2003), time developing online courses (Dahl, 2003;Zhang, 1998), and training on online course development (Feist, 2003). These concerns suggest that the most impactful way to "disrupt" education is to be cognizant of what a single faculty member can reasonably do for students when online instruction competes with offline demands, and to provide more money, time, and resources to ensure that faculty are supported in this work.
Neither LMS significantly transformed our classrooms, but we witnessed how a more user-friendly, intuitively designed LMS can enhance learning opportunities for students and cause fewer headaches for instructors. There is no doubt that even more classrooms will be converted to virtual real estate in the future. The act of teaching and learning will increasingly involve interfacing with an LMS in some shape or form. Thus, this technology will consume a larger portion of our lives than in the past. More research is needed to document how LMS in the "mobile web" era shape student learning, and to what end. Studies that compare what LMS are trying to achieve with how learners actually use them would be particularly useful to understand how students in online courses become active participants in their own learning.