The Effect of McGraw-Hill Education Connect on Students' Academic Performance

This study aimed to investigate the effect of using McGraw-Hill Education (MGHE) Connect on students' academic performance. It examined the effects of MGHE Connect on the course letter grade, pass rate, retention rate, total score, and final exam score in addition to the proportions of having more A and B grades in the course. The study used a posttest-only control group design. The 95-student sample was selected by sampling four sections using a simple one-stage cluster method. Then two sections were randomly assigned as a treatment group (N=45), in which using MGHE Connect Chemistry was required to complete course assignments, quizzes, and tests, and the other two assigned as a control group (N=40) in which students completed the course conventionally. The study used ordinal logistic regression, binomial logistic regression, and multiple linear regression. The study indicated that when controlling the effect of students' CGPA, there was no significant difference in the odds of having higher letter grades between the experimental treatment and control groups. Also, the experimental group did not significantly have a higher proportion of A and B grades compared to the control group. The course pass rates and the retention rates in both groups differed insignificantly. The total course score and the course final exam score did not differ significantly. The study findings of students' academic performance metrics in the study indicating no statistically significant positive effect of the MGHE Connect on the student academic performance. Keywords—Adaptive learning, blended learning, LearnSmart, textbook technology supplements, student performance, course management system, online learning


Introduction
In today's technology-driven societies, higher education institutions, especially at the tertiary level, are adopting blended learning approaches to enhance student learning and improve student performance [1]. The blended learning model combines traditional classroom teaching and an e-learning system [2][3][4]. In this way, instructors combine the advantages of online education with traditional face-to-face teaching and thus enhance student learning [2], [5][6][7].
The concept of blended learning has recently received much attention, and online teaching is increasingly a part of course delivery [2], [6]. As digital millennial college students are more comfortable with the online learning, blended learning became the trend in teaching models and learning styles and gradually replaced the traditional teaching and purely online learning [2], [6], [8][9][10].
Many researchers believed that blended learning results in higher learner performance than either distance learning or traditional face-to-face instruction alone [11][12][13]. For example, Deschacht and Goeman found that the introduction of blended learning led to higher exam scores and a slightly higher course pass rate [14]. Sarabadani and Berenjian found that students who accessed the online modules in blended learning, compared to those who did not, scored higher in the examination [15]. Gambari et al. found a significant advantage for blended learning over e-learning and traditional classroom teaching [16]. As Obiedat et al. concluded, blended learning generally has many advantages over traditional and e-learning ways of learning, and it has a positive effect on students' study process [17].
Several meta-analyses provided different accounts of the positive impact of blended learning on students' performance. For example, Bernard et al. reviewed 232 studies published from 1985 to 2001 and found that interactive blended learning had a minuscule impact on students' achievement compared to traditional classroom instruction (g= .055) [18]. On the other hand, Means et al. examined 45 studies published from 1996 through July 2008 that compared the effectiveness of both purely online learning and blended learning with traditional face-to-face learning [19]. The study indicated that, on average, studies of the combinations of pure online and blended learning performed better than learning through the face-to-face method solely (the mean effect size was +0.20, p < .001). Also, the study found that while the mean effect size for the purely online versus face-to-face contrasts was not significantly different from 0 (g+ = +0.05, p = .46), the mean effect size of the blended versus face-toface contrasts was significantly different from zero (g+ = +0.35, p < .0001) [19]. Many researchers concluded that the reported effects of blended learning on students' achievement were inconsistent and mixed [1][2], [9], [12], [14], [20].
Regardless of the mixed findings of blended learning effects, increasing demands from educators have led the proliferation of technology supplements provided by famous textbook publishers such as McGraw-Hill, Belford/St., Martin's, and Pearson [21]. These textbook technology supplements became prolific in higher education as complementary tools to assist student learning [1], [22].
McGraw-Hill Education (MGHE) developed a leading course management system called "Connect," which is being used extensively in higher education institutions around the world to enhance traditional methods of teaching and improving students' performance [1]. MGHE defined Connect as "a digital learning environment that integrates assignments, grading, and course content, making the entire course experience seamless while delivering better outcomes for students and educators (p.4) [23]." Part of the MGHE Connect platform is LearnSmart, which is an adaptive learning tool that can be used on top of fully online or on-campus courses at any university [6], [24]. Students can study each chapter of the course using LearnSmart designed to improve students' understanding of course contents through its online interactive plat-form [6]. In order to prove the effectiveness of its Connect platform on student performance, case studies research was conducted in collaboration with MGHE in different institutions in the United States. The studies concluded that Connect had a positive impact on all aspects of student learning, such as performance, pass rate, retention rate, exam score, and final course grade [1], [23].
Despite the studies conducted in collaboration with MGHE, there is little independent empirical evidence and little peer-reviewed work which was assessing the efficacy of MGHE Connect LearnSmart tool, and the results of investigations have been mixed [6], [24][25]. While there is a great deal of research to assess the effectiveness of such technologies on student learning and performance, not much is being conducted on platforms provided by publishing companies like MGHE [1]. The MGHE's platforms are available for a range of courses and are used throughout the world.

Past Independent Studies on MGHE Connect
The author in [24] examined the effectiveness of MGHE Connect LearnSmart as adaptive learning technology, as compared to the traditional teaching method, in an undergraduate management information course. The sample comprised 102 students in the Spring semester 2019 Information Technology course in the College of Business; using LearnSmart was optional for extra credit in the class. Students were exposed to the same classroom lectures, textbooks, assignments, and exams, and the only difference was the degree of LearnSmart usage. The study conducted a correlation analysis between LearnSmart usage and course grades. It also tested the difference between the course grades of the group that used LearnSmart and the group that did not. The study indicated no significant relationship between LearnSmart usage and test/course grades. Also, it indicated no significant differences in course grades between the group which used LearnSmart and which did not. The study concluded that LearnSmart did not enhance learning beyond traditional methods, and it could be efficient for the learning process rather than for the outcomes.
The study [1] examined the effectiveness of MGHE Connect on improving student grades in a precalculus course at a university in a Gulf Cooperation Council (GCC) country. The study focused on the use of Connect in Mathematics and Statistics to investigate the effect on student grades. The course was delivered traditionally in the Spring semester of 2012. In Fall 2012, the course used Connect online assignments (homework and quizzes) allotting 10% of the course grade. The participants were 30 students in each course. The study indicated a significant positive correlation between online assignments and course grades. The study also concluded that the use of Connect improved students' overall course grades.
The authors in [25] conducted a study aiming to test the utility of LearnSmart as a commercially available adaptive e-learning tool in a massive undergraduate Psychology course. The study tested the relationship between LearnSmart usage and academic achievement while controlling for five psychological predictors of academic success (intellectual ability, conscientiousness, openness to experience, need for cog-nition, and epistemic curiosity). The study focused on two assessment pieces: quizzes and the final exam. The sample contained 467 students enrolled in a first-semester Psychology course 1A, and 542 students enrolled in a second-semester Psychology course 1B. LearnSmart usage was optional in Psychology 1A and obligatory in 1B. The study indicated that students who made use of LearnSmart performed significantly better on the assessment tasks compared to non-users in the two courses. Also, the study indicated that the extent to which students made use of LearnSmart was a more reliable predictor of academic performance than the four personality variables that previously had seen as related to academic outcomes.
The study [26] aimed to examine adaptive teaching and learning techniques, and if they increase student scores, pass and retention rates, and increase efficiency for both students and faculty. The study used data over three years regarding student and instructor outcomes in an Introduction to Computing course, a vast multi-section course, with an average of more than 90 sections per semester, 26 students per section, and 45 instructors per semester. The study utilized two adaptive learning tools: SimNet and MGHE Connect, in addition to Brightspace D2L as a learning management system (LMS). The study indicated that the means of the midterm exam and final exam in sections with adaptive learning were higher than without adaptive learning. Also, the students' pass rate with adaptive learning was higher by almost 10%. The rate of A's and B's increased when adaptive learning was used, while D's and F's declined.
The author in [21] examined MGHE Connect LearnSmart online textbook supplement and its effect on students' exam performance in an interpersonal communication course. The study sampled 62 students distributed in two groups (n=29 for treatment/n=33 for control). The study used a group comparison, posttest-only experimental design to compare the effect of LearnSmart usage on student exam performance. The courses were delivered on the same day by the same instructor in the same room and with identical content being covered. The two groups were compared across some demographic characteristics such as sex, classification/year, the program of study (majors vs. nonmajors), the average number of absenteeism during the semester, and the GPA of students before the semester. The study also measured students' perceptions of and satisfaction with the LearnSmart. The study findings indicated that the exam performance did not significantly improve for students using an online resource. However, results did demonstrate a trend of positive effects based on students' satisfaction and perceptions.
The study [27] tested whether over 600 students' exam scores were associated with the use of textbook technology supplements (TTS's). Three different textbooks and different TTSs (MGHE Connect LearnSmart, PsychPortal, and Aplia) were used. Three of the introductory Psychology classes participated in the study for over three years. The instructor required the online study aid use as part of the class and made it count towards the course grade (extra points awarded on the final exam). The study indicated that TTS use significantly predicted final exam scores. Also, online homework increased students' performance on exams and quizzes if students regularly utilize the site.
The authors in [28] examined the effectiveness of MGHE Connect LearnSmart for student learning and outcomes in undergraduate anatomy and physiology courses in six schools. The study used a pretest-posttest experimental design in which 323 and 264 students were assigned in control and treatment groups, respectively. The study compared scores improvement (posttest score minus pretest score), grade distributions, and retentions between the experimental and control groups. The study indicated no statistically significant differences in score improvement, grade distributions, and retention rates between the experimental and control groups. However, students showed positive perceptions. The majority of students liked LearnSmart and found it useful. The study concluded that LearnSmart might have other impacts on students that were not well reflected in the students' outcomes.
The author in [29] examined four specific aspects (time, completion, metacognitive data, and program-generated student score) of MGHE Connect LearnSmart, and the potential effects these aspects might have on student assessment performance. The study examined the relationships between LearnSmart use and student quiz and exam scores. The study used a large section that included 193 students and was taught by one instructor. While the LearnSmart was not required for the course, it was available to all students. The results indicated statistically significant relationships between the module student score, module completion, total time spent on all LearnSmart exercises, and total average percent completion with students' exam scores. The percent completion and time were more reliable indicators than others of exam performance.
The past studies covered the use of MGHE Connect and LearnSmart in various disciplines such as management information, interpersonal communication, psychology, computer science, precalculus, biology, and anatomy and physiology. They used various research methodologies, such as correlational, experimental, and survival methods, and indicated mixed results of MGHE Connect effects on students' performance.
This study aims to investigate the effect of using MGHE Connect on students' academic performance. It examines the effects of MGHE Connect, if any, on students' course letter grades, having A's or B's course grades, total course score, course final exam, course pass rate, and course retention rate. Based on the literature review, and the objectives the study tries to achieve, the study would test the research hypotheses stated below.

Research Hypotheses
H1a: MGHE Connect has a statistically significant effect on having a higher letter course grade.
H1b: After controlling the effect of CGPA, MGHE Connect still has a statistically significant effect on having a higher letter course grade.
H2: MGHE Connect has a statistically significant effect on having more A's and B's course grades when controlling CGPA.
H3: MGHE Connect has a statistically significant effect on improving the course pass rate when controlling CGPA.

H4
: MGHE Connect has a statistically significant effect on improving the course retention rate when controlling CGPA.
H5: MGHE Connect has a statistically significant positive effect on the total course score when controlling CGPA.
H6: MGHE Connect has a statistically significant positive effect on the course final exam when controlling CGPA.

4
Materials and Method

Research design
The study used a posttest-only control group design in which an experimental group and a control group would be compared on a posttest measure. This design assumes that groups before the introduction of the experimental manipulation would be substantially equivalent due to the random assignment of individuals to conditions [30]. MGHE Connect was used as a platform required to prepare and review for classes in addition to complete all course activities such as homework, assignments, quizzes, and exams by the experimental group students. On the other hand, the control group students studied the course and completed all activities and requirements traditionally. All quizzes, assignments, and tests given were identical. The main difference was using MGHE Connect, as an interactive learning platform, in the experimental group and the traditional face-to-face style in the control group.

Participants
The participants in this study were undergraduate introductory course to Chemistry students at Yanbu Industrial College (YIC), a technical college in the western region of Saudi Arabia. Nine sections of Chemistry 101 were offered in the Fall semester of 2018, with an average of 22 students in each section. The sampling method used was a single-stage cluster in which four sections out of the nine were selected to participate in the study. Then, two out of the four selected sections were assigned randomly to be a treatment group, and the other two to be a control group. The cluster sampling method was used due to practicality since it was difficult in the credit-hour system to control students' assignments in a specific time slot or a section. The head of the General Studies Department, offering the course, allocated randomly two assistant professors to participate in the study and assigned randomly to each instructor one section from the treatment group and another one from the control group.
The total sample size of the study was 95 students distributed based on the group they are assigned in and their sections, as shown in table 1. 40 students (42.1%) were in the control group while 55 students (57.9%) were in the treatment group. Instructor#1 (sections 1&9) taught 55 students (58%) while in-structor#2 (sections 7&8) taught 40 students (42%).

The instrument
This study used MCGH's Connect Chemistry as an experimental treatment. MCGH's Connect creates a digital learning environment integrating assignments, grading, course content, and learning resources [23]. It was fully compatible with the textbook used in Chemistry 101 and satisfied the course learning objectives and outcomes. MGHE Connect includes an eTextbook with quizzes, practice problems, in addition to an interactive learning tool called LearnSmart. Connect also enables students to access educational resources available by the instructor, including Power-Point presentations, audios, and videos, in addition to the eBook [1]. Each chapter in the MCGH's Connect eTextbook has an associated online LearnSmart module, which instructors assign to cover the course content and for formative and summative assessment [24]. LearnSmart presents questions based on the course content, and students are required to answer and assess their confidence in the correctness of their answer on a four-point scale (I know, Think so, Unsure, or No idea) [24]. Then, LearnSmart adjusts the difficulty level of subsequent questions based on the accuracy of each response, and students have to review the material and answer questions correctly before they can progress further [24]. All LearnSmart questions are mapped to the eTextbook chapters. Also, LearnSmart can take the student directly to the eTextbook content and highlight in yellow the content providing the answer, and students can repeat the work until they master the material [24]. Along with the course textbook, a test bank was provided in which all questions are multiple-choice questions mapped to the content [24].

4.4
The procedure At the beginning of the fall 2018 semester, the YIC's General Studies Department contacted the MGHE regional representative to set up accounts for instructors and provide the necessary training and instructions to create the course, activate accounts, developing assignments and quizzes, mentoring students' activities, and navigating through the system. At the beginning of the second week of the semester, the experimental group students were introduced to MGHE Connect and were provided access, which was free of charge. Students in the experimental group were provided with the necessary training and instructions on how to use Connect and benefit from it. They were requested to complete their assignments, quizzes, and exams through the Connect platform. Also, they were directed and encouraged to use the LearnSmart to prepare and review their classes. Students in the control group were requested to complete all course activities traditionally. During the course, students in both groups took four quizzes, three assignments, a midterm, lab work, and a final exam. All quizzes, assignments, and tests were equivalent. At the end of week 15 of the semester, all students sat for an equivalent final exam and received their grades. The study was approved by and conducted following the ethical guidelines of the research committee of YIC's General Studies Department. Participants received an explanation of the study and its procedures and gave their informed consent before its commencement.

Measures
The measurement of students' academic performance, based on the literature, was assessed using the course letter grade, the proportion of having A&B grades, the total score, the course final exam score, the course pass rate, and the course retention rate [6], [23], [31]. All these measures were used as dependent variables. The standard course grading scale used in YIC and this study was as follows: The letter grades were coded from 1 for F grade up to 9 for A+ grade as an ordinal variable called "Grade." Also, collapsed categories of letter grades would be used for further data analysis. The total score would be out of 100 points and was composed of: 5% homework, 10% quizzes, 15% midterm exam, 35% lab work, and 35% final exam (total=100). The final exam score was out of 35 points and was scored based on a proctored exam given at the end of the semester. The course pass rate was calculated by taking the ratio of the number of students who passed the course with any grade above C to the number of students taking the final exam [32]. The course retention was calculated as defined by MGHE: "student retention measures the percentage of students who complete the course compared to the number who enrolled at the beginning of the term (p.5)" [23]. For the retention analysis, enrollment was measured using the college standard: all students registered minus those who had not shown after the first two weeks, whether because of never appearing or dropping within the time [33]. The independent variables used in the study were group, coded as (1=experimental/0=control), and Cumulative Grade Point Average (CGPA out of 4.0) representing the CGPA that a student had prior starting the experiment and was extracted from the administration system.

4.6
The statistical analysis The study used IBM SPSS version 20 and Microsoft Excel to utilize various statistical tools. Descriptive statistics such as means, standard deviations, bar graphs, line graphs, scatter plot graphs in addition to 95% CI were used to present and compare variables. Regression analyses were used to examine the statistically significant effects of the independent variable and covariate on the dependent variable. The cumulative odds ordinal logistic regression with proportional odds was used when the dependent variable was ordinal such as course letter grade. When the dependent variable was dichotomous such as having A's or b's grades or not, passing the course or not, and retaining the course or not, the binary (binomial) logistic regression analysis was used. In the case of continuous dependent variables such as the total course score and the course final exam, the study used the ordinary least square multiple regression analysis. In all cases, the assumptions of the concerned analyses were verified before interpreting the results.
In ordinal logistic regression, the proportional odds assumption was assessed by a full likelihood ratio test comparing the fitted model to a model with varying location parameters using the Chi-Square test (p>.05). The model's goodness of fit was tested using the deviance of the goodness-of-fit test (p>.05) and Pearson goodness-of-fit test (p>.05). The model effect size (the Pseudo R-Square) was reported as Cox and Snell, Nagelkerke, and McFadden. The statistical significance of the model was measured by testing the difference between predicting the dependent variable using the model and the intercept-only model (p<.05). The effect of the independent variable was measured using the respective significance level of coefficient estimate (p<.05) and 95%CI of the estimate.
In binomial logistic regression, the linearity of the relationship between the continuous independent variable and the logit of the dependent variable was examined using the Box-Tidwell procedure [34]. The outliers were detected using the standardized residual value (<-.2 or >.2). Hosmer and Lemeshow's goodness-of-fit test was used to examine if the model fits the observed data (p>.05). The Nagelkerke R2 statistic measured the effect size.
For the linear multiple regression analysis, the independence of residuals assumption was assessed by a Durbin-Watson statistic (<10), The linearity of the relationship between the dependent variable and independent variables were examined by using the scatter plot of residuals (residuals should form a horizontal band). Also, the relationship between the dependent variable and the covariate variable was examined using "partial regression plots" (should show a linear relationship). Homoscedasticity was examined by plotting the scatterplot between the predicted values and studentized residuals. Collinearity Tolerance (>.1) and VIF <10) indicated that multicollinearity assumption was met. Potential outliers were examined using the studentized deleted residual (SDR) (<3). The leverage points and the max value of Cook's distance were assessed using the saved values in SPSS (<.2 and <1, respectively) [35]. The assumption of normality was examined by using P-P Plot & Q-Q Plot, in such the points should be aligned along the diagonal lines. R2 measured the model effect size, and its value was assessed according to Cohen in [36]. The overall model statistical signifi-cance was assessed based on ANOVA F-test (p<.05). The effect of the independent predictor on the dependent variable was assessed using the respective regression coefficient B and its 95%CI, and its significance was assessed using a t-test (p<.05).

The effect of MGHE connect on student letter grades
The effect of MGHE Connect on a student letter grade could be depicted in figure  1, showing how grades were distributed in each group of treatment and control groups.    figure 2c, the distributions were clearer that the proportions of above-average grades in the treatment group were higher than the control group. On the other hand, the average grade and below proportions were lower than the control group. Figure 3 shows how grades were distributed within each group:

Fig. 3. How Percentages of Grades Were Distributed in Treatment and Control Groups
The figure showed that A grades (A+ and A) in the treatment group were higher by 27 points compared to the control group. In contrast, D and F grades were about 8% less in the treatment group compared to the control group. In sum, all figures showed a potential positive effect of the treatment on the course letter grades.
Testing H1a: MGHE Connect has a statistically significant effect on having a higher letter course grade. To examine if the positive treatment effect on the student grades (A+, A, B+, B, C+, C, D+, D or F) was statistically significant, a cumulative odds ordinal logistic regression with proportional odds was conducted using the group variable as a predictor and grade as the dependent variable. The proportional odds assumption was assessed by a full likelihood ratio test comparing the fitted model to a model with varying location parameters, and it was satisfied, χ2(7) = 5.59, p = .589. The deviance goodness-of-fit test [χ2(7) = 5.59, p = .589] and Pearson goodness-of-fit test [χ2(7) = 5.39, p = .612] indicated that the model fitted to the observed data well. The Pseudo R-Square measures were: Cox and Snell (.367), Nagelkerke (.406), and McFadden (.182). The final model significantly predicted the dependent variable better than the intercept-only model, χ2(1) = 6.06, p < .05. The treatment had a statistically significant effect on having a higher grade, Wald χ2(1) = 5.86, p < .05. The odds of having a higher grade in the treatment group were 2.648, 95% CI [1.204, 5.824] times of the control group, χ2(1) = 5.86, p ≤.05. So, the odds of having a higher grade on the dependent variable in the treatment group was nearly twice to triple of the control group. This finding indicated that the students using MCGH's Connect were more likely to get higher letter grades in their course compared to the control group students.
The finding presented above indicated a significant positive effect of the treatment on student course grades and supported research H1a. However, because the sampling method used was not optimum, and the research design did not have a pretest measure to control the groups' equivalency, students' CGPA, before starting the study, which might be influential, was used to control. Thus, further investigation was conducted to reassure if the significant effect of the treatment on students' grades, shown above, would still be there if the effect of a student CGPA was statistically controlled.
An exploring graph analysis in figure 4 shows the means and 95% CI of CGPA of students in both control and experimental groups. The figure showed that students assigned in treatment and control groups were not equivalent in their CGPAs. The 95% CI showed that the mean of students' CGPAs in the treatment group was significantly higher than that in the control group. This bias in the prior CGPAs would be a real threat, and it should be controlled. So, the study would restate the hypothesis about the effect of the treatment on the course letter grade and restrict it to be after controlling the effect of CGPA.
Testing H1b: MGHE Connect still has a statistically significant effect on having a higher letter course grade when controlling the effect of CGPA. To examine if the treatment has a statistically significant effect on the students' grades after controlling the effect of student CGPAs, a cumulative odds ordinal logistic regression with proportional odds was run. The student grade using the YIC grading scale (A+, A, B+, B, C+, C, D+, D, and F) was the dependent variable. The group variable (treatment/control) was the main factor and CGPA as a covariate. There were 88.5% of cells with zero frequencies, which is something expected when the ordinal logistic regression model contains a continuous covariate variable. A large number of cells with zero will not affect the parameter estimation, it might affect the overall goodness-of-fit statistics, and that is why they should be treated with suspicion in such a case [37]. The proportional odds assumption was examined using a full likelihood ratio test comparing the fitted model to a model with varying location parameters, χ2 (14)  .017] times that for the control group, χ2(1) = .17, p =.684, indicating that after controlling the effect of CGPA, the odds for the treatment group to get better grades were lower than the control group. However, this negative effect of the treatment on grades was statistically insignificant and research H1b, stating that MGHE Connect still has a statistically significant effect on having a higher letter course grade when controlling the effect of CGPA was not supported.
In sum, the findings indicated that the experimental treatment was not effective in improving the course letter grades when controlling the effect of prior student's CGPA. Consequentially, the results suggested that the effect of the treatment could be attributed to the students' prior CGPAs instead of the treatment itself.

5.2
The effect of MGHE connect on having more "A" and "B" grades The A's and B's grades were distributed in the treatment and control groups, as shown in figure 5. The graph showed that the proportion of students getting A's and B's in the treatment group was 26% higher compared to the control group. However, this hypothesized statement should be statistically tested to check if that finding was statistically significant or not.

Fig. 5. The Distributions of A's & B's Grades in the Treatment and Control Groups
Testing H2: MGHE Connect has a statistically significant effect on having more A's and B's course grades when controlling CGPA: Binomial logistic regression was performed to investigate the effects of group and CGPA on the likelihood of getting an A or B grade in the course. The linear relationship of the CGPA with the logit of the dependent variable was assessed via the Box-Tidwell procedure [34]. Based on this assessment, the CGPA was found to be linearly related to the logit of the dependent variable. There were two standardized residuals with values of 2.74 and 2.89 standard deviations, and those cases were kept in the analysis. The goodness-offit for the model was tested using the Hosmer and Lemeshow's test, and it was statistically insignificant [χ2(8) = 6.997, p = .537] indicating that the model was not a poor fit.
The logistic regression model was statistically significant, χ 2 (2) = 36.40, p < .001. The model explained 48.0% (Nagelkerke R 2 ) of the variance of having an A or B grade and correctly classified 74.4% of cases. Sensitivity was 73.0%, specificity was 75.6%, positive predictive value was 71.0%, and the negative predictive value was 77.3%. In the model, only CGPA was statistically significant (as shown in Table 3), and the treatment group did not show a statistically significant effect on the odds of having As or Bs in the course. So, research H2 stating that MGHE Connect has a statistically significant effect on having more A's and B's course grades when controlling CGPA was not supported, and there was no evidence that the treatment promotes more A and B grades in the course.

The effect of MGHE connect on the course pass rate
The pass rates in treatment and control groups were as shown in figure 6. The figure showed that the treatment group is higher in the pass rate by 8% compared to the control group.

Fig. 6. The Pass Rates of Treatment and Control Groups
Testing H3: The MGHE Connect has a statistically significant effect on improving the course pass rate when controlling CGPA: Binomial logistic regression was performed to examine the effects of group and CGPA on the likelihood of passing the course. The linear relationship assumption of the CGPA with the logit of the dependent variable was assessed via the Box-Tidwell procedure [34], and the CGPA was found to be linearly related to the logit of the dependent variable. There was one standardized residual with a value of -2.16 standard deviations, which was a case that did not pass the course successfully while its CGPA was above 3.0. So, that case was kept in the analysis. The Hosmer and Lemeshow's goodness of fit test was statistically insignificant [χ2(8) = 7.92, p = .441], indicating that the model was a good fit. The logistic regression model was statistically significant, χ2(2) = 18.27, p < .001. The model explained 30.2% (Nagelkerke R 2 ) of the variance in passing the course and correctly classified 80.5% of cases. Sensitivity was 95.2%, specificity was 31.6%, positive predictive value was 82.2%, and the negative predictive value was 66.7%. The CGPA was statistically significant, as shown in Table 4, while the treatment did not show a statistically significant effect on passing the course. So, the finding did not

The effect of MGHE connect on the course retention rate
Retention rates in treatment and control groups were as shown in figure 7. The figure showed that the retention rate in the treatment group is 6% lower than the control group. This descriptive analysis showed a negative effect of the treatment on the course retention rate. However, the statistical significance of such observation would be more evident once the hypothesis is tested, as shown below.

Fig. 7. The Retention Rates of Treatment and Control Groups
Testing H4: MGHE Connect has a statistically significant effect on improving the course retention rate when controlling CGPA: Binomial logistic regression was performed to check the effects of the treatment and CGPA on the likelihood that the student retains the course. The linear relationship of the CGPA with the logit of the dependent variable (retention) was assessed via the Box-Tidwell procedure [34], and the CGPA was found to be linearly related to the logit of the dependent variable. There were two standardized residuals with values>3 standard deviations, and they were kept in the analysis. The model was a good fit since the Hosmer-Lemeshow goodness of fit test was statistically insignificant [χ2 (8)  and correctly classified 88.4% of cases. Sensitivity was 98.8%, specificity was 23.1%, positive predictive value was 89.0%, and the negative predictive value was 75.0%. The CGPA was statistically significant, as shown in Table 5, while the treatment was not. Thus, research H4 stating that the MGHE Connect has a statistically significant effect on improving the course retention rate when controlling CGPA was not supported, and the finding indicated that the treatment did not statistically significantly affect the course retention rate.

5.5
The effect of MGHE connect on the student total score when controlling the CGPA The means of the course total scores in the treatment and control groups are shown in figure 8.  Figure 9a shows that residuals form a horizontal band, which means that the relationship between the total score dependent variable and independent variables most likely was linear. Also, the relationship between the total score variable and the covariate variable (CGPA) using "partial regression plots" was linear, as shown in figure 9b. The scatterplot in figure 9a also showed that there was homoscedasticity. Also, there was no multicollinearity issue in the model [Collinearity Tolerance=.81 (>.1) and VIF=1.23 (<10)]. When the studentized deleted residual (SDR) was examined to check if there were potential outliers, only one case was found to have SDR>3 standard deviations, and it was kept in the analysis. Also, when the leverage points were checked, only one case was found to have the value .20, indicating that no case had a problematic leverage value. The max value of Cook's distance=.21 (<1), which means that there was no influential case. The assumption of normality was examined by using P-P Plot & Q-Q Plot, as shown in figures 10a and 10b. The P-P Plot and Q-Q Plot showed that although the points were not aligned perfectly along the diagonal lines, they were close enough to indicate that the residuals were normally distributed. R 2 for the overall model was 44.6%, with an adjusted R 2 of 43.2%, which means a large size effect. The model significantly explained and predicted the total score F (2,79) = 31.86, p < .001. The CGPA significantly predicted the total score t=7.392, p<.001, B=9.930, 95%CI [7.256,12.604]. The treatment group was statistically insignificant [t=-.496, p=.622, B=-1.209, 95% CI (-5.562-3.345)]. These findings did not support research H5 stating that the MGHE Connect has a statistically significant positive effect on the total course score when controlling CGPA and indicated that there was no effect for the treatment on the course total score. Regression coefficients and standard errors can be found in Table 6.   Figure 12a showed that when the studentized residuals were plotted versus the predictive values, the residuals formed a horizontal band indicating that the relationship between the dependent variable and independent variables was linear.
Also, figure 12b showed that the relationship between the dependent variable and the independent covariate variable (CGPA) using "partial regression plots" was linear too. Moreover, the assumption of homoscedasticity can be examined using the Plot presented in figure 12a. The scatterplot showed that there was homoscedasticity. Also, there was no multicollinearity issue in the model [Collinearity Tolerance=.81 (>.1) and VIF=1.23 (<10)]. When the studentized deleted residual (SDR) was examined to check if there were potential outliers, only one case had SDR>3 standard deviations. Also, when checking for leverage points, one case only had a value of .20, indicating that no cases had a problematic leverage value. The max value of Cook's distance=.062 (<1), which means there was no influential case in the dataset. Figures 13a and 13b showed P-P Plot & Q-Q Plot to examine the normality assumption.  The P-P Plot and Q-Q Plot showed that although the points were not aligned perfectly along the diagonal, they were close enough to indicate normality. As the multiple regression analysis is reasonably robust against deviations from normality, the study can accept this result and conclude that the assumption of normality was satisfied.
R 2 for the overall model was 39.8%, with an adjusted R 2 of 38.3%, a large size effect. The model significantly explained and predicted the final exam score F (2,79) = 36.115, p < .001. The CGPA significantly predicted the final exam score t=6.759, p<.001, B=5.368, 95%CI [3.787,6.949]. In contrast, the experimental group was not statistically significant [t= -.629, p=.531, B=-.832, 95% CI (-3.465-1.801)]. The findings did not support research H6 stating that the MGHE Connect has a statistically significant positive effect on the course final exam when controlling CGPA. Regression coefficients and standard errors can be found in Table 7.

Discussion
This study examined the effect of MGHE Connect on students' academic performance. Past studies used different methods to examine the effectiveness of online adaptive learning systems, such as the MGHE Connect platform. Most of them were correlational research [1], [24][25][26][27], which could not establish a cause-and-effect relationship while some were experimental [21,28]. This research was an experimental one in which randomization and control groups were employed. However, because of the practicality issues related to the registration regulations, the study relied on cluster sampling in the random selection of the study sample and random assignment into experimental and control groups. The study used clusters as an alternative, even though cluster sampling was not the best technique to control potential error sources [38][39]. Using a student's CGPA as a covariate variable in data analysis for statistical controlling was the solution, especially once it was clear that the two groups were not equivalent in their prior CGPAs.
In this study, when the data were analyzed to check if using the treatment improved the course letter grades or not, the descriptive statistics and graphs showed a positive effect of the treatment on grades. These findings were consistent with most other research findings showing the positive effect of adaptive learning tools and blended learning approaches. However, such findings reflected how dangerous it would be to reach conclusions based on descriptive statistics. For example, the study [26] concluded that adaptive learning improved course grade, pass rate, having more A's and B's grades, having fewer D's, and F's grades. Nevertheless, all of their conclusions were driven based on descriptive statistics and not on statistical tests.
In this study, when the measures of descriptive statistics and the odds of having better grades in the experiment group compared to the control group were tested statistically, the difference was statistically significant in favor of the treatment group. Nevertheless, when the analysis statistically controlled the CGPA effect, the unique contribution of the experimental treatment effect on the students' letter grades became trivial and statistically insignificant. This finding showed how important it is to control research circumstances. Lack of controlling confounding variables could lead to false conclusions. Thus, the study concluded that using MCGH's Connect was not significantly sufficient to improve the course letter grades, and this finding was confirmed by [28].
The study [28] concluded that the differences in grade distributions when students used MGHE LearnSmart compared to not using LearnSmart were not statistically significant. However, it was not enough to test whether the course letter grades were different in the two groups or not. Letter grades' scale interprets grades on both sides around the midpoint differently. For example, having more As or Bs grades are positively interpreted while having more Ds or Fs grades would be interpreted negatively. So, when the test deals with grades multinomially and examines if grades' distribution in the treatment group is equal to or different from the control group, the test result will not help to derive a conclusion that there was a positive or a negative effect on grade distribution. The test at best would tell that either the two distributions were identical or different, and it would not tell how different they were. In such a case, ordinal logistic regression would be very vital to answer a question such as how likely it is for grades to increase incrementally from the lowest grade up to the highest grade in the experimental group comparing to the control group.
The study findings indicated that, after controlling the CGPA effect, there was no statistically significant difference in the proportions of A's and B's grades in treatment and control groups. Also, there was no statistically significant difference in the course pass rates between the treatment group and the control group. These findings contradicted what [23] and [26] concluded. Unlike this study's finding, the conclusions of [23] and [26] were driven based on descriptive statistical analyses and did not rely on statistical tests. The finding of an insignificant effect of the treatment on the pass rate also means that there was no treatment effect on having fewer D's and F's grades. The reason is simply that the pass rate was defined as having a C grade or above, and what would be left is to have D's or F's grades or what we can call it failing rate. Only the study [26] stated that its finding indicated fewer D's and F's grades when adaptive learning was used, and again that was without a statistical test.
Also, the study indicated that there was no difference in the retention rates between the experimental treatment group and the control group. This finding agreed with what found in [28] but disagreed with what [26] concluded. Unlike the descriptive than correlational studies, but still, it was not determinant. Also, the lack of randomization of individuals' assignments into the groups weakened controlling the bias. Moreover, the measures of dependent variables such as the course total scores and final exams were derived from instructor-made tests and were not validated. Finally, it is essential to interpret research findings very cautiously. Failing in proving the effectiveness of a program does not mean that the program eventually is ineffective. Simply it means that there is no enough evidence in the study to prove the program effectiveness.
Further research is suggested to replicate such a study in different disciplines and circumstances. Also, it is suggested to investigate other aspects of the MGHE Connect advantages, not only the impact on academic performance. Student's performance is critical to evaluate the effectiveness of any program but not only the one. Student's learning is very complicated, and academic performance is just one aspect of the process. Further research is suggested to pay more attention to the confounding variables and to emphasize independent research as the world is heading towards technologydriven teaching environments and needs proper investigation of the reliability and validity of such learning tools.