Characteristics of Students Who Opted In to Use the Boost Mobile App as an Educational Support Service

This exploratory study aimed to investigate the characteristics of students who opted in to use Boost, an automated student-support mobile app, and compare them to characteristics of Boost nonusers. Boost integrates with a learning management system (Canvas) and provides support services aimed at improving student behavior and success. At the start of the spring 2019 semester at Indiana University, instructors were invited to opt in for Boost to be available to their classes. Instructors who did so invited their students to use Boost. Our multivariate analysis of variance compared those who opted in for automated support with those who did not (n = 158 courses). Findings reveal that optins were further along in their studies and slightly lower performing than their peers who did not opt in. A profile of Boost users will help university administration, student support services, and instructors make data-informed decisions on the optimal use of Boost.

improvement in critical care management in patients with asthma and diabetes who received guided management strategies through education via SMS (Goodarzi et al., 2012;Lv et al., 2012;Zamansadeh et al., 2017). Healthcare is arguably one of the more advanced disciplines in the way technology has been implemented for use in interventions; as such it provides an ideal touchstone for studies in higher education.

Literature Review and Research Questions
With the rapid pace of technological advancement and online education being one of the fastest growing segments of higher education in the United States (Seaman et al., 2018;Ginder et al., 2019), technology is blurring the lines between "traditional" and "nontraditional" students. Most often, the age of a student has been a defining characteristic of traditional and nontraditional students (Bean & Metzner, 1985). Traditional postsecondary students are commonly those who are recent high school graduates, between 18 and 23 years of age when first enrolled (Chartrand, 1990(Chartrand, , 1992Jinkens, 2009), and from medium-high socioeconomic status families (Bradley et al., 2008;Choy, 2002).
Nontraditional students, in contrast, are those who are 24 years of age and older (Chartrand, 1990(Chartrand, , 1992Jinkens, 2009). Online students enrolled at an institution of higher education are more likely to be 25 or older, married with children, attending school part-time, and working full-time (Campbell & Wescott, 2019;Jaggers & Xu, 2013); they have academic challenges that differ from those of their traditional, on-campus counterparts, such as time and location constraints.
Nontraditional students are perceived as having considerable barriers to higher education. The convenience of online classes provides greater access to higher education, particularly for students who balance family, work, and school responsibilities. This also includes disadvantaged students, such as low-income, minority, and first-generation college students. These students often have more access and resource constraints due to family commitments (Hiltz & Shea, 2005), work responsibilities (Dutton et al., 2002;Hiltz & Shea, 2005), financial limitations (Leasure et al., 2000), and geographical barriers (Dutton et al., 2002) compared to their nondisadvantaged peers. Clearly, nontraditional students have always required flexibility, and before online classes, this need was met with night classes, weekend seminars, and correspondence courses (Cavanagh, 2012). However, many students may be attracted to online classes for other reasons, such as to participate in intercollegiate athletics or to make time for a social life. Students seize the affordances an online education has to offer (Beqiri et al., 2010;Bocchi et al., 2004). However, availability and convenience do not translate to success as an online learner.

Self-Regulated Learning
The concept of SRL emerged from Albert Bandura's (1997) self-efficacy theory. Self-efficacy was developed as part of the broader social learning theory, which progressed into social cognitive theory (Zimmerman & Schunk, 2003). Zimmerman (2001) found parity between SRL theory, with its roots in self-efficacy and social cognitive theory, and a myriad of other psychological theories, for example motivation, achievement and social learning theories.
One of the widely cited SRL models developed over the last two decades proposed by Zimmerman (1998Zimmerman ( , 2002 describes SRL as a student's "self-generated thoughts, feelings, and actions that are planned and cyclically adapted to the attainment of personal goals" (Zimmerman, 2000, p. 14). These self-generated thoughts occur throughout a cyclical three-phase sequenced routine: (1) forethought, (2) performance or volitional control, and (3) self-reflection. Two major steps within each of the three phases have been identified (Zimmerman, 1986).
In the forethought phase, learners perform task analysis, which consists of goal setting and strategic planning. They should also be developing self-motivation beliefs, including self-efficacy, outcome expectations, intrinsic interest value, and learning goal orientation. Novices in the forethought phase are found to be reactive learners. Because they lack goal setting, these learners compare their performance to the learning of other students (Zimmerman, 1986).
The performance or volitional control phase consists of self-control, which includes imagery, self-instruction, attention focusing, and task strategies, and self-observation, which includes selfrecording and self-experimentation. In this second phase, learners use strategies and monitor their performance. Experts in this domain agree that self-monitoring is the crucial element for successful SRL (Corno, 1986;Corno & Mandinach, 1983;Mace & Kratochwill, 1988;Nelson, 1977;Schunk, 1989;Shapiro, 1984).
In the final phase of self-reflection, learners reflect on their performance and evaluate the outcome.adapt. They engage in self-judgment, which consists of self-evaluation and causal attribution, and self-reaction, which encompasses self-satisfaction, affect, and adaptive or defensive behavior .
Students who have stronger SRL skills are typically self-starters: They display persistence on learning tasks, are confident and strategic in overcoming problems, and are self-reactive to task performance outcomes (Zimmerman & Schunk, 1994). Indeed, SRL requires learners' active effort in monitoring their own study habits (Bjork et al., 2013;Fernandez & Jamet, 2016). Studies have emphasized that SRL is not a fixed trait but rather a skill that can be developed and refined through instruction, role models, experience, and practice by applying SRL strategies (Schunk, 2005;Zimmerman et al., 2015).

Learning Analytics
Learners with underdeveloped SRL skills benefit from support to improve their ability to self-regulate their learning. However, researchers have shown that even skillful learners manifest deficiencies in their SRL skills (Pressley & Ghatala, 1988 and benefit from support. Increasingly, learning analytics and learning analytic dashboards (LADs) have been used as an intervention to provide support to all students (Kennedy et al., 2014). LADs are intended to provide near-real-time feedback to the student and other stakeholders (Few, 2006). Recently, researchers at the University of Iowa revealed that students who frequently monitored their LAD had significantly higher grades on assignments and tests than students who did not (Van Horne et al., 2018). However, in their literature review, Bodily and Verbert (2017) asserted that minimal research has addressed the ways students are using LADs and how to increase student use. More research is needed on the perceived and actual effects of LADs on student behavior, achievement, and skills.
While LADs represent a promising technology for improving students' understanding of their learning behaviors, these tools are passive by design and rely on students seeking their benefits and interpreting their visualization. Another method of providing more active support is through autoreminders or nudges. Nudging platforms can monitor students' schoolwork, and rather than render these data in a visualization, a nudge can proactively contact a student through a mobile notification. For example, in the student support mobile app Boost, Indiana University students can receive proactive nudges reminding them about imminent assignment deadlines to help them stay on track (Motz et al., 2020). But even while a mobile notification system can intervene to support self-regulation directly, it shares a limitation with LADs that students must make some initial effort to use the service (in this case, to install it on their phones).

Research Question
The benefit of a dashboard, a mobile auto-reminder app, or any system or strategy that is intended to support SRL is determined by the types of students who volunteer to use these tools. While there is an opportunity for instructional systems to develop SRL strategies more directly, any implementation relies on its use by students who have room to improve their SRL strategies. However, there is limited research on the types of students who opt in for support services. The purpose of this research study was to investigate the characteristics of students who opted in to use Boost and compare them with the characteristics of their non-Boost-using classmates (Indiana University eLearning Design and Services, 2018). Boost is a mobile app that uses the Canvas application programming interface (API) and integrates with the Canvas learning management system (LMS). The Boost app is designed to help busy students keep track of their schoolwork. Students configure the kinds of push notifications they want to receive in the app. For example, students can select (1) unsubmitted assignment due date reminders, (2) instructor announcement notifications, (3) daily assignment digest, (4) calendar events, and (5) display Canvas To Do list.
As a mobile app that integrates directly with the LMS, Boost is able to clearly identify students who have taken steps to install the app and activate it within the course, making it an ideal platform for identifying the types of students who seek out support tools for SRL. Developing an understanding of students who opt in to use the Boost app will help instructors and administrators as they work together to improve student support services. Aggregate properties of Boost users were used to answer the research question: What are the characteristics of students who opt in to receive automated support, compared with those who do not opt in?

Participants
At the start of the spring 2019 semester at Indiana University, instructors were invited to participate in a research study using a smartphone app developed by Indiana University designed to help students keep track of their schoolwork in the learning management system (LMS). This no-cost app named "Boost" was downloadable from the iOS and Android app stores. From the mobile app, students authenticate to the LMS and select individual course(s) for which to receive reminders via push notifications about their coursework. During this spring 2019 semester, only courses taught by instructors who had explicitly opted in to have Boost available would be operational within the app. Invitations to participate were sent via various university listservs, including those for teaching centers and learning technology units, and a global announcement via the LMS. Instructors were eligible to participate if they were teaching a for-credit course with an active, published course on the Canvas LMS (Coates, 2008).
Instructors of 738 courses with published Canvas sites opted in to have Boost available to their students, with an average enrollment of 40.5 students in each course. Courses in which fewer than two students signed up for Boost or fewer than two students did not sign-up for Boost during the spring 2019 semester were excluded from further analysis (which filtered out a large number of low-enrollment courses), leaving a final sample of 158 courses, with an average enrollment of 47.1 students.
All analyses in this study were performed at the course level, contrasting the aggregate properties of students who volunteered to use Boost (who downloaded, installed, and logged in to Boost during the spring 2019 semester) with the aggregate properties of students who did not volunteer to use Boost. Least squares weighting was used to account for different enrollment sizes in these aggregate summaries at the course level (see Data Analysis, below).

Procedures
Instructors who opted in to have Boost available to their students were provided with a verbal script and an email template to invite students to download the Boost mobile app, both of which were approved by the Indiana University Institutional Review Board (IRB). Up to three additional IRBapproved invitation emails containing instructions on how to access the mobile app were sent from boost@iu.edu, to students who had not yet decided to opt-in or opt-out of Boost. All students completed their course in the usual fashion, according to instructor and discipline, but were allowed to use the app if they desired (no incentives were provided, other than the possibility that Boost might help them avoid missing deadlines).
After downloading the app, students authenticated via a single sign-on service to connect with the university's instance of Canvas and to consent to participate in this research and have their data analyzed. Upon providing consent, students then configured the app by specifying the kinds of push notifications they would receive in the app. However, students who did not use Boost provided no such consent, which is why all analyses in this study were performed at the aggregate course level, rather than at the individual student level.

Data Collection
Data that already existed in institutional databases were retrieved from various data sources, primarily the student information system and the Canvas data warehouse. No individually identifiable data were returned in the database queries for comparing notification tool users to their peers; only aggregate class-level means and percentages were analyzed for Boost users and Boost nonusers. The class itself was de-identified in the analysis data set (no section numbers, course numbers, or campus information was included) to eliminate the possibility of deductive disclosure of student information. No student was identifiable in the data under analysis for this contrast, nor were the study data able to be mapped back onto individual students.

Data Analysis
To investigate the research question, course-level aggregate data from Indiana University were consolidated and analyzed using IBM SPSS statistical software (version 26.0). Data were analyzed using a between-subjects study design approach (Charness et al., 2012) wherein a generalized linear model was selected, as it is traditionally the primary method used for the analysis of count data (McCullagh & Nelder, 1989;Wood, 2006). A multivariate analysis of variance (MANOVA) was used in the data analysis as it is an accepted test suited to investigating between-group differences (Field, 2013). We applied weighted least squares regression to make the distribution of the number of students in the data approximate the distribution of the number of students in the population from which the sample was drawn. Weights are a function of observed independent variables included in the model. Data analysis began by determining Wilks's lambda criterion (Wilks, 1932). A one-way MANOVA was calculated examining the effect of Boost (use or no use) on average age, percentage undergraduate, percentage female, percentage married, percentage instate, percentage international, percentage first generation, percentage White, percentage Asian, percentage underrepresented, and percentage multiracial variables. Lambda was not significant, λ(11, 304) = .965, p > .05, suggesting that these demographic and academic history variables account for a relatively small percentage of variance in whether students opt in to auto-reminders, as expected from SRL theory.
Quantitative data were analyzed both descriptively and inferentially. Descriptive statistics were used to describe the basic features of the data. In this study, a MANOVA was used to compare the characteristics of Boost users with the characteristics of Boost nonusers.

Descriptive Analysis
We report the means and standard deviations for continuous variables and percentages observed in each course. Across the courses that met the inclusion criteria, an average of 6.3 students volunteered to use Boost (13.3%). With two levels of an independent variable (Boost, or no Boost), the aggregate demographic values and percentages were analyzed as dependent variables in a linear model, weighted by the number of students making up each observation at the course level.

Sociodemographic Characteristics of Boost Users
Descriptive statistics for each of the sociodemographic study variables are presented in Table 1. Results show the percentage of Boost users who were White was lower than the percentage of Boost nonusers who were White, F(1, 314) = 7.516, p < .05. This was primarily driven by a significantly higher percentage of Asian students using Boost, F(1, 314) = 5.635, p < .05, many of whom also contributed to a considerably higher percentage of international students using Boost, F(1, 314) = 4.201, p < .05. The demographic profile of students who opted in to use Boost skewed largely in the direction of students who were minorities but who were not considered underrepresented. Descriptive statistics for each of the study variables related to educational background are presented in Table 2. There was a significant difference between average number of credits for which each student was enrolled during the Spring 2019 semester in Boost users and average number of credits earned by each student as of the start of the Spring 2019 semester in those who did not use Boost, F(1, 314) = 3.794, p < .05, and similarly, the Boost users were enrolled in more credits than those who did not use Boost, F(1, 314) = 3.840, p < .05. SAT scores (which were imputed in the case of students who entered college with alternative test scores such as the ACT) were higher among Boost users than those who did not use Boost, F(1,314) = 5.016, p < .05, despite a directional trend for Boost users to have a slightly lower prior grade point average (GPA) than those who did not use Boost, F(1,314) = 3.598, p = .059. *p < .05 differences between groups.

Discussion
The purpose of this study was to compare the sociodemographic characteristics and academic performance of students who opted to use the Boost app with those of students who chose not to use the app. The students who opted to use Boost were more likely to be Asian and international and had higher scores on college entrance exams than their classmates who opted not to use Boost. However, students who used the app did not show signs of outperforming their peers who did not use the app. If anything, there was a trend that Boost users had slightly lower prior GPAs than Boost nonusers, despite having earned significantly more credits in college at the time of the study. It would seem that, in this study, students who volunteered to receive automated reminders were those who had traditionally outperformed their peers, but after experience in college without such performance had realized that they could benefit from automated support. A critical component of academic success is SRL (Broadbent & Poon, 2015), which requires forethought, performance, and self-reflection (Zimmerman & Schunk, 1989). In 2019, the Boost app was a new tool on campus that was available to only a subset of the university. Recruitment came from selected faculty who agreed to invite their students to participate. Boost was also a new tool for the faculty. In addition to being recruited by a faculty member in class, students in participating classes also received three email reminders inviting them to participate. Students were ultimately left to decide if they would download the Boost app and how they would implement it. The results of this study show that students who ignored or decided against using the app had earned fewer college credits than their counterparts who chose to use the app. Arnold and Pistilli (2012) asserted that students early in their college career are not often aware of the behaviors or necessary actions needed to be successful.
Further, in terms of anticipated performance, there is evidence that the lowest performing students are often the most inaccurate at predicting their prospective academic performance (Kruger & Dunning, 1999). There is reason to believe that students who need additional help are those who pass it up. Academic advisement centers and faculty can recommend students download Boost, particularly at-risk and lower achieving students. However, students who are insecure about their knowledge or ability are also less likely to seek help (Karabenick & Knapp, 1991). Providing a nonthreatening environment to set up Boost may increase the usage, particularly for at-risk, firstgeneration, and underrepresented populations.
Results from the study indicate that Boost users had registered for and earned more credits than those who chose not to participate. It can be argued that students who opted to use Boost had a better understanding of the challenges of keeping on top of their homework and understood the behaviors and actions necessary to be successful. It is also possible that this subset of students from our study possessed characteristics of SRL study behaviors and were in what Zimmerman (1986) called the forethought phase, where learners strategically plan and set learning goals. However, the current study did not collect any self-report data to support this hypothesis.
Boost users as a group had a significantly higher percentage of Asian and international students than the group who did not use Boost. Asian students in the United States typically suffer from a "model minority" stereotype-where they are viewed as hard working and academically gifted (WongLai, Nagasawa, & Lin, 1998). Social pressure from perceptions of this stereotype may lead Asian students to become particularly sensitive to identity threats in both traditional and online classes (Lagier, 2003;Wang, 2007). Language barriers may also exacerbate these challenges, such that when their academic performance dips, Asian students may have more difficulty seeking out assistance (Yeboah & Smith, 2016). For these students, Boost may provide uniquely beneficial, nonthreatening, automated support to help them stay on top of their coursework.
Other demographic groups could also clearly benefit from automated support. Our results suggest institutions may need to invest extra effort to get tools such as Boost into the hands of underrepresented minorities, nontraditional students, or incoming freshmen (among others). In contrast to Asian students who may adopt Boost because of massive social pressure toward academic achievement, other segments of the student population may need to be convinced of the challenges of an academic workload and the necessity of additional support. This need to be convinced is at the heart of the difficulties faced when training college learners to adopt SRL strategies and skills, helping students reach the forethought phase before they experience academic challenges that are difficult to overcome.
This study was not without its limitations. By design, this study had a small sample size (only open to those instructors who opted in), a limited student population (only one university), and a onesemester duration of the study, which are all factors that limit generalizability. While our aggregate course-level data provide useful information for general understanding, more can be learned by investigating educational support services, specifically mobile app interventions employing learning analytics, at the individual level. Nonetheless, this generalized understanding lays essential groundwork for such studies.
Our research goal focused on better understanding the characteristics of students who did and did not opt in for Boost and how they might differ. To further this research, we propose to explore how students interact with Boost, specifically, their push-notification tapping behaviors. For example, we plan to explore the types of notifications students are more likely to tap on and the frequency with which notifications are tapped, identify the point in the semester at which students are more likely to tap on notifications, and compare assignment submission rates for tapped and nontapped notifications. Further, it would be useful to understand students' self-regulation study behaviors, why they opted to use Boost, and if they perceived Boost as useful.

Conclusion
Learning analytics as support for student self-regulated study behaviors is a rapidly evolving research domain in student success scholarship. With the continued growth of online learning, a growing concern for the rising costs in higher education, and student retention efforts, this research has a critical application to the higher education landscape. Educational intervention research can influence and impact student success. The use of real-time automated services to support student SRL behaviors and academic achievement can improve student retention. More research is needed in this area.