The Pursuit Of Excellence: 3rd-Party Rankings And Positioning Of Online Programs Through Quality And Value

Third-party ranking agencies (e.g., US News and World Report, Center for Online Education, CollegeChoice.net) produce rankings that are a popular publicly accessible search option for potential students as they sort online degree options. While higher education administration is keen to the possibility that most or all ranking systems risk methodological flaws, potential students are still on board with rankings and seem significantly steered by their results (Barkhorn, 2014). Monitoring rankings has become a positioning consideration for academic institutions. The disparity amongst ranking factors across time and across assessor, along with the mismatch between ranking methods used and suggested methodologies from research on higher education begs the question: which attributes are colleges and universities supposed to leverage? This study reports a fully mediated SEM using data from online student feedback across multiple quality touch points: faculty, LMS and course on student loyalty as mediated by perceived value.


INTRODUCTION
he Babson Survey Research Group's 2017 Distance Education Enrollment Report measures online class participation as approximately 29.7% of all college students, wherein roughly half take a combination of online and live courses, while the other half are exclusively online students. Over 4,000 public colleges and universities (67.8%) leverage the online platform as a medium for course delivery, possibly to the detriment of live enrollment, which is in decline nationwide (Allen & Seaman, 2017).
Demand of online course delivery is at an all-time high with millions of students enrolling in courses each term; likewise, there are more than 28,000 accredited online degree programs prepared to meet the demand of the projected 15% increase in post-secondary enrollment by 2025, according to Projections of Education Statistics to 2025 (Hussar & Bailey, 2017, p. 24). With such a highly populated, competitive market, how do post-secondary institutions cut through the clutter to reach potential students? Colleges and Universities can position themselves by type: public, private, not-for-profit, for-profit; delivery options: exclusively online, traditional with online presence; or perhaps by accreditation: DOE-recognized accreditation, or external accrediting entity. Positioning strategies might also include affordability, quality, service quality scores, faculty quality, or post-graduation student success. Research suggests a 5-factor model of quality in higher education: learning effectiveness, access, student satisfaction, faculty satisfaction and cost effectiveness (Moore, 2002).
Ranking agencies such as US News and World Report, Center for Online Education, or CollegeChoice.net produce rankings for colleges and universities that are a popular publicly-accessible search option for potential students as they sort and rank options. While the vast majority of higher education administration is keen to the possibility that most or all ranking systems are inherently flawed, potential students are still on board with rankings and are significantly Lafuente-Ruiz-De-Sabando, Forcada and Zorilla (2017) suggest that while universities should strive to create value for students and other stakeholders, they must also strive to "make the perceptions (image or reputation) that its stakeholders have of it, its services and its brands, more positive" (p. 15). The authors further note that these two areas of focus (value creation and perception management) represent separate aspects of the marketing effort that "do not always go hand in hand" (p. 16). As with most other organizations, a complicating factor for higher education institutions is the fact that they must focus on value creation and perception management for multiple stakeholdermarkets, each with differing ideas of value and differing perceptions. We seek to better understand the dynamics between the antecedents and consequences of the value of the online degree from the perspective of the student, but acknowledge that value and (importantly) perceptions of value (image/reputation) may be different from the perspective of other stakeholders (perhaps reflected in the various college guides?). Since college guides frequently change methodology (as noted above) and even incorporate judgments of competitors, those rankings should also be considered carefully as surrogate measures of image/reputation for various stakeholders.
This study uses online student feedback across multiple data touch points to assess the relative strength of some of the most popular considerations undertaken by students searching for that perfect online college fit. Quality of faculty, learning management system (LMS) and course content are assessed as drivers of student perceived value, and in turn, loyalty.

VALUE IN ONLINE HIGHER EDUCATION
Categorizing the quality dimensions of online learning is vital as learners are exposed to multiple quality touch-points prior to, during, and after the completion of online learning (Ehlers, 2004). Recently, research has advanced into highly-focused, quantifiable measures of online higher education program/course quality as it pertains specifically to the preparedness and quality control of faculty (Parscal & Riemer, 2010), and rigor of instructional design, web design and course presentation (Chao, Saj & Tessier, 2006). In addition, numerous quality assurance programs have been launched to assess overall online program, course, faculty, and student interaction quality: Quality Matters, Educause, Institute for Higher Education Policy's Benchmarks for Success, Khan's Eight Dimensions of e-Learning Framework, and the Sloan Consortium's Five Pillars of Quality.
Customers calculate perceived value as the trade-off of gains versus sacrifice (Zeithaml, 1988). Students compare their sacrifices (time, effort, tuition and additional costs) to anticipated benefits (convenience, utility, degree). Extensive research posits that increases in value stem primarily from either decreases in sacrifice or increases in gains (Monroe, 1990). As students base their college choice on 3 rd party rankings, their main value assessment includes quality considerations pertaining to academic (both faculty and course-specific) and technological support and delivery quality assessments. This study primarily focuses on the student rating of faculty quality, quality of the LMS, and the quality of the content of the course as drivers of the student value perception. Perceived value is assessed here using Zeithaml, Parasuraman & Malhotra (2005) assessment of students' evaluations of price, convenience and control (Respondents ranking Very Poor to Excellent: the cost of online classes, overall convenience, overall value in exchange for your tuition and effort). This value measure closely reproduces CollegeChoice.net's Value ROI ranking (tuition vs. graduation pay) (2017).

Academic Quality
Most 3 rd -party ranking institutions combine faculty and course quality to reach an "Academic Quality" assessment.  (2017), mainly based on faculty degree level. However, degree level and type of degree do not perfectly represent quality of instruction. Faculty quality is measured here using the institution's 12-item, 1-factor formalized student evaluation tool. Student feedback on faculty evaluation surveys is a longstanding method (Carrell & West, 2010). Logically, the higher the students perceive faculty quality, the higher gain they perceive as they assess value. We hypothesize that overall value will be significantly, positively impacted by positive student rankings of faculty quality.
H1: Faculty quality will be a positive, significant indicator of student perceptions of value of their online degree.

Course Quality
Course quality is defined as an objective aggregate a student forms for each of their courses depending on relevance to their degree, profession or interests, and their impression of how well they liked the course (Harvey & Green, 1993). CollegeChoice.net ranks colleges and universities on curricular quality (2017). The site doesn't disclose particulars on curricular quality assessment. Course content quality is measured here using formalized student feedback of course quality using a 3-item quality scale (Would you recommend this online course to another student? Do you feel that the online program is an adequate alternative to an on-campus classroom?) We hypothesize that overall value will be significantly, positively impacted by positive student rankings of course quality.

H2:
Course content quality will be a positive, significant indicator of student perceptions of value of their online degree.

Copyright by author(s); CC-BY 22
The Clute Institute

LMS Quality
The learning management system (LMS) is the web-based systematic framework that delivers and manages instructional content, identifies and assesses learning goals, and collects and presents data for learning assessment (Lonn & Teasley, 2009). Research is widespread on the practical importance of optimal LMS utilization. Students place the most importance in LMS as it lends itself to time savings (Hanson & Robison, 2004), and achievement of learning goals. Blackboard, Canvas, WebCT are examples of relatively popular LMS technologies readily employed by colleges and universities as infrastructure for course delivery. These and other systems organize and enhance the online classroom experience.
Formalized quality measurement of the LMS varies. Research has utilized the System Usability Scale (SUS) (Orfanou, Tselios & Katsanos, 2015) the TAM (Ros et al. 2014) to assess LMS adoption. E-service quality broadly assesses the extent to which a website or web-based interface such as an LMS facilitates efficient and effective delivery (Zeithaml et al. 2005). The authors introduce E-S-QUAL, a 22-item revision to their SERVQUAL measure, along with four newly-identified dimensions: efficiency (Can tasks be completed quickly?), fulfillment (Is feedback delivered as promised?), system availability (Is the site always available?), and privacy (Information privacy is protected within courses). Oldfield and Baron (2000) suggest that student perceptions of quality should include requisite, acceptable and functional elements to fulfill their essential, desired and utilitarian educational needs, respectively.
Studies have shown that high levels of service quality positively influence customer satisfaction (Rao, Goldsby, Griffis & Iyengar, 2011), competitive positioning (Swaid & Wigland, 2012), and loyalty (Hallowell, 1996). US News & World Report assesses Online Offering Quality with a "Student Support and Technology" ranking (Brooks & Morse, 2017). Interestingly, the other 2 ranking sites did not include an exclusive metric for online quality. This study utilizes E-S-QUAL to assess the quality of the LMS. We present the following hypothesis regarding LMS quality as a significant antecedent to student perceptions of value in the online higher education setting.
H3: LMS quality will be a positive, significant indicator of student perceptions of value of their online degree.

Full Mediation of Perceived Value on the Faculty, LMS and Course Content Quality-Loyalty Linkage
Students' intentions to remain loyal capture their attitude or behavioral intent to remain a customer of a particular service provider through repeat purchases (Hallowell, 1996). Research has determined that loyalty is an outcome of service provider image, perceived service quality, switching costs, as well as satisfaction (Aydin & Ozer, 2005) as well as perceived value (Alves,201). A substantial stream of research links service quality perceptions to service customers' loyalty intentions through the mediating effect of perceived value (Zeithaml et al. 2005;Swaid & Wigand, 2012). The full mediation of perceived value between antecedents and student loyalty is hypothesized as follows ( Figure 1):

Study
Self-report data was collected from undergraduate and graduate students at a 4-year, regionally accredited, public, click and mortar university located in the Southeastern US. The sample collected was 127 (out of 941 students).
Approximately 70% of the sample indicated taking solely online classes for their purely online degree, while the remainder took a mixture of both live and online classes to complete their degrees. The majority of students ranged in age between 23 and 47. Students completed web-based surveys reflecting their perceptions of faculty quality, global course content quality, E-S-QUAL, perceived value of their degree and student loyalty measures.

Results
Structural equation modeling using AMOS 24 was used to assess the impact of each of the 3 antecedents of perceived value, with 2 of the 3 (Course Quality and LMS Quality) showing positive, significant linkages (p<.001). Faculty Quality as a driver of student perceived value had a positive, but non-significant linkage (sig. .889). H2, H3, and H4 were found to be significant and in the expected direction ( Table 2). The R 2 for course and LMS quality accounts for approximately 47% of the variance in perceived value. In turn, perceived value accounts for 31.5% of the variance in student loyalty. Finally, the full mediation model was assessed using Bootstrapping ( Table 2). The direct model of antecedents to loyalty at a 95% Confidence Interval show the direct effect of course quality (.000) and LMS quality (.000) on loyalty is not significant, while the indirect effects of course quality on perceived value (.571) and LMS quality (.231) and the direct effects of perceived value on loyalty (.561) shows significance and support for a fully mediated model (GFI: .986, TLI: .959, RMSEA: .067) (Figure 1).

DISCUSSION AND FUTURE RESEARCH
Perhaps the most interesting finding is the lack of significance of faculty ratings on perceived value. The findings presented here suggest that the quality of the LMS and the course itself are far more influential on student perceptions. One possible explanation that future research could explore is the relevance of a course (required or elective) could moderate the faculty -value linkage. Students may view the faculty differently in required vs. elective courses.

Copyright by author(s); CC-BY 24
The Clute Institute Students may also simply weight the transcript take-away as more important in their value equation. Great faculty may be a bonus, but if they meet a qualification minimum, their styles and delivery quality may be indirect on their value perception. Future research could investigate when and where faculty ratings fit in the student assessment of value.
Course content quality, however, showed significance on value, which is one of the more interesting findings of this research. The finding highlights a disparity between a variable that higher education holds as vital, while 3 rd -party ranking sites typically include nothing to indicate course quality. This difference is an obvious disconnect between what universities use to position their program quality and what students view as important through ranking sites.
One area where 3rd-party sites and university resource allocation seem to align is the LMS. The greatest implication for practice relating to technology is that information technology infrastructure is crucial to universities and colleges seeking to improve student perceptions of value and rankings of online programs. The survey questions relating to how fast and reliably pages and sites load and work highlight the importance of information technology infrastructure. Institutions anticipating variability in growth of demands on their infrastructure need to consider scalability. Colleges and universities need to be cautious about privacy and data protection if they want to do well with privacy measures.
Vendors for Internet services and LMS vendors should be selected carefully and care should be given to vendor relationships. Given the potential impact of technology on both 3rd-party rankings and student value perceptions, LMS developers and other software providers with significant customer bases in higher education would do well to pay attention to these technology performance issues. Additionally, vendors and developers of LMS products should bear these impacts in mind when developing new features or improving features to support their higher education customers who need or want to perform well in external rankings and to improve student evaluations of the value of academic programs. Back et al. (2016) found that students valued ease of use in an LMS. Universities and colleges considering changing LMS products or vendors would be wise to pay attention to how those products may support these efforts given their possible impact on 3rd-party rankings. These items technology measures also emphasize the importance of faculty support and training on LMS products, features and tools so that faculty and course designers can create courses so that they strengthen efficiency items to both enhance students' perceptions of value and to support efforts to do well in external rankings.
Colleges and universities seeking to improve their performance on the efficiency measures should also consider the support they provide to online students. The quality of mobile support of LMS products and features can impact several items in the efficiency and system availability areas which may influence rankings. While it may not be within the control of particular colleges and universities, variability of Internet connectivity and IT infrastructure in communities or regions could influence some of these factors.
A relatively new area for 3 rd -party ranking sites not assessed in this study is "Peer Ranking". The opportunity for universities here is enormous should they already have high brand name recognition. As institutions assess each other, it is entirely possible that they assign higher quality rankings simply based on name recognition. The inclusion of such a peer-ranking is a plus for any large institution, which for whatever reason (academic excellence, sports teams, notable researchers, or alumni) has high brand recognition. Smaller, less well-known institutions, however, may view peer rankings as a strategic drawback. US News and World Report weighted peer rankings as 20% in 2017 (Brooks & Morse, 2017). It would be interesting research the peer ranking process in future research: are universities any more insightful than the potential students they criticize for relying on university brand name as a decision-making heuristic?
Finally, managers responsible for the design and marketing of online programs within higher education must be market oriented (Hammond, Webster & Harmon, 2006;Zebal & Goodwin, 2012), and cognizant of the multiple stakeholdermarkets as they make decisions surrounding value-creation and image/reputation management (Lafuente-Ruiz- De-Sabando et al. 2017). Further research is needed to better understand similarities and differences in the dynamics between value, perceptions of value, and reputation regarding online degree programs from the perspective of multiple stakeholder-markets within higher education.