A Text Mining and Statistical Approach for Assessment of Pedagogical Impact of Students’ Evaluation of Teaching and Learning Outcome in Education

Technology-enhanced learning (TEL) is now at the heart of teaching and learning process in many higher education institutions (HEIs). Today, educators are faced with the challenges of pedagogically specifying what tools, methods, and technologies are used to support the teachers and students, and to help maintain/sustain a continuous education and practices. This study shows that there is an opportunity in the use of (educational) datasets derived about the teaching and learning processes to provide insights for fostering the education process. To this effect, it analyzed the students’ evaluation of teaching (SET) dataset ( $n=471968$ ) collected within a higher education setting to determine prominent factors that influences the students’ performance or the way (TEL-based) education is being delivered, including its didactical impact and implications for practice. Theoretically, the study employed a mixed methodology grounded on integration of the Data-structure approach and Descriptive decision theory to study the rationality behind the students’ evaluation of the teaching and performance. This was done through the Textual data quantification (qualitative) and Statistical (quantitative) analysis. Qualitatively, the study applied the Educational Process and Data Mining (EPDM) model (a text mining method) to extract the different sentiments and emotional valence expressed by the students in the SET, and how those characteristically differ based on the period and type of evaluation they have completed (between 2019 to 2021). For the quantitative analysis, the study used a multivariate analysis of covariance (MANCOVA) and multiple pairwise comparisons post-hoc tests to analyze the quantified information (average sentiment and emotional valence) extracted from the SET data to determine the marginal means of effect the different SET types and evaluation period have on the students’ learning outcomes/perception about the teaching-learning process. In addition, the study empirically discussed and shed light on the implications of the main findings for TEL-based Education, particularly implemented by the HEI during the analyzed periods. The scholastic indicator from the study shows that while the flexible digital models or instructional methods are effective for continuous education, innovative pedagogies, and teaching transformations. It also, on the other hand, serve as an incentive for more robust research that idiosyncratically look into their implications for the students’ learning outcomes and assessment done in this study.

innovative pedagogies, and teaching transformations. It also, on the other hand, serve as an incentive for more robust research that idiosyncratically look into their implications for the students' learning outcomes and assessment done in this study.
INDEX TERMS Text mining, teaching assessment, TEL-based education, learning models, students evaluation, sentiment analysis, educational innovation, higher education.
However, in education, while the TEL-based methods and models have shown to be promising for improving the teaching and learning processes for the stakeholders, for instance, the use of Learning Analytics (LA) and Learning Design (LD), Interactive Whiteboards, Virtual Reality (VR) and Augmented Reality (AR), Massive Open Online Courses (MOOCs), Canvas, Blackboard, Computer-supported Collaborative Learning, Blended Learning, and Flipped Classrooms [1], [14], [22], [23], [24], [25], [26]. On the other hand, there are discourses, both in theory and practice, on the issues and challenges, or what TEL-based methods pedagogically offers toward the transformation or heightening of the educational processes, particularly as it concerns the flexibility, sustainability, and scalability of the new and emerging (technology-based) educational models or instructional approaches [13], [15]. Those challenges include to mention but a few; issues that deal with the social mandate or lack of local capacity to design and build specialized educational technologies, to lack of coalition between the existing educational models and operational policies of the different institutions in respect to the educational labor market [27]. There is also the problem of digital divide or gap (otherwise allied to institutions or persons lacking access to the digital technologies and platforms who are potentially excluded from the vast benefits and opportunities to TEL) [28], [30], [31], [32], to inadequate skills, training, and use of the different technologies by the teachers and students, and inability of educators to leverage the information (educational datasets) that are being recorded and stored at an unprecedented rate in the databases of the different institutions to support the decision making processes and performance strategies [7], [27], [33], [34], [35], [36], [37], [38], [39].
Consequentially, we note that the aforenoted issues and challenges is an indicator for the development of educational models or methods that supports provision of ''innovative pedagogies'' for teaching and learning, that not only integrates the TEL-based initiatives, but can be instructionally used to bridge the gap between the modern and traditional models of teaching, whilst paying attention to the new mechanisms or practices for achieving the much-needed ''flexibility, scalability and sustainability of education''. Indeed, one of the most pertinent ways through which the said goals can be achieved is through conceptual analysis and leveraging of the insights drawn from the readily available educational datasets that explores the learning patterns and performance of the teachers and students [40], [41], for an improved educational practice as done in this study.

A. THE RATIONALE OF THE STUDY
To describe the pedagogical impact, assessment, and evaluation of teaching in context of this research, this study uses information (educational dataset) drawn from the teachersstudents learning outcome and experiences (SET) to analyze and understand the impact of the flexible digital models for learning, students' satisfaction and assessment, and how those can be used to determine/develop useful insights and strategies for an enhanced learning and outcomes [4], [7], [12], [13], [42], [43], [44]. The use of educational datasets (e.g., SET) to support the teaching and learning process has become one of the recent and important discourses that have emerged both in the existing literature and in practice [3], [27], [28], [37], [41], [45], [46]. Pedagogically, the dissemination of ''student-generated data'' can be used to provide an increased performance and for addressing the different challenges associated with the TEL-based education [13], [15], [47], [48]. Thus, this study proves that there is a need for innovative methods for extracting useful (education-based) information from the datasets recorded and stored about the students' evaluation of teaching/performance, to help transliterate them into actionable plans for the educators and/or TEL-based education models and curriculum design.
To this end, the study applied a two-step (mixed) methodology grounded on synthesis of the Data-structure approach [49] and the Descriptive decision theory [50], [51] to study the rationale behind the decisions that the learners are disposed to make by means of the textual data quantification (qualitative approach) and statistical analysis (quantitative approach). This was done using the students' evaluation of teaching (SET) dataset collected in a higher education setting. Qualitatively, we applied the EPDM model (a text mining method) [52] to determine the extent or level of impact (sentiment and emotional valence) of the different comments provided by the students in the SET, and how those may characteristically differ based on the type and period of SET evaluation. For the quantitative analysis, we utilized the extracted information (i.e., the average sentiment and emotional valence) to determine the marginal mean of effect that the evaluation periods (between 2019-2021) and type of evaluation (Students-to-Instructor, Student-to-Students, and Overall recommendation) has on the students' evaluations and outcome, and how those may statistically differ based on the studied period and SET type.
The main research questions of this study are as follows: 1) How can we analyze the (educational) dataset captured about the students' evaluation of teaching (SET) to determine prominent factors that influence the students' performance? 2) Does the students' evaluation and recommendations vary based on the period and type of SET evaluation completed by the students? 3) How can the results and outcome of the study be used to provide actionable insights towards improvement of the TEL-based education? The main contributions of the study are as follows: 1) It defines a data-structure approach based on the descriptive decision theory to understand the students' perception of the teaching-learning processes and outcomes within the higher educational settings.
2) It provides a two-step mixed method (through a qualitative and quantitative lens) applied to determine the influential factors and marginal means of effects upon how the SET evaluations impacts and differ based on the type and periods of evaluation. 3) It shows the usefulness of the Text mining technique (textual data quantification) in understanding the intensities or impact of the comments provided by the students in the SET. 4) It empirically discusses how educational datasets extracted about the teaching/learning processes can be analyzed and used to provide actionable insights and solutions to the TEL-based education including its implications for practice. The rest part of the paper is structured as follows: recent and relevant studies in the topic, particularly as it concerns the TEL-based education models and students' evaluation of teaching (SET) are discussed in the Background Information (Section II). Section III consists of description of the research methodology, the SET instrument, data sampling, and experimental setup. The data analysis and results are presented in Section IV. The study empirically discussed and shed light on implications of the results and findings in Section V, including the limitations and directions for future research. Conclusions of the study is provided in Section VI.

II. BACKGROUND INFORMATION A. STUDENTS EVALUATION OF TEACHING AND ASSESSMENT
Existing studies have looked into the cogency or impact of the students' evaluation of teaching (SET) as an indicator of teachers' performance and assessment [41], [52], [53], [54], [55], [56]. For instance, Boring [53] notes that SET has lately been used not just as a tool to assess the teaching performance of the teachers/instructors, but have also led to studies or researches that look into the qualities or teaching dimensions that the students find important in their teachers [57]. The several studies have shown that a lot of the time, the higher institutions of learning (HEIs) greatly rely on the outcomes of the SET not only for improving or refining the various learning activities and curriculum, but also, are used to define the academic performance of the teachers [48], [53], [55], [58]. In this current study, we note that an important key aspect of the SET should be on its coalition with the educational ecosystem, and how such (educational) dataset can be used to understand the teaching/learning practices, including the identification of steps that can be taken to improve the dynamics of the teaching/learning processes [27], [40]. In this vein, there is a necessity to tenaciously study the concealed benefits of SET in education, and how the underlying information that are derived from such tools can be used to support the decision-making processes, and drive the educational ecosystem forward. The authors believe and prove that studying the comments provided by the students when completing the SET instruments can allow educators to identify the students' learning pathways, experiences, and expectations towards an enhanced learning and teaching practices. This can be didactically done by transliterating the extracted sentiments and emotions expressed by the students in the completed SET into meaningful insights in connection to the teachinglearning process, and to discover the prominent factors that influences the way they rate their teachers and learning outcomes [40], [57], [58].

B. TECHNOLOGY-ENHANCED MODELS FOR LEARNING IN CONTEXT OF SET
The instructional and practical challenges related to ''technological infrastructures'' for learning faced by both the educators and students has led the university' leaders to reinvent learning options that combine the virtual and faceto-face learning modalities [59], [60], [61], otherwise allied to the TEL-based education [15], [43]. Whilst the TEL or hybrid models for learning have shown to be more flexible and caters for continuous education, especially during the recent global pandemic [13], [62], [63]. It on the other hand, could potentially lead to relaxed curriculum. Pedagogically, the hybrid (TEL-based) educational models can be said to come with both opportunities and challenges that the various HEIs need to address and adopt in order to achieve the wide VOLUME 11, 2023 benefits (instructional and technical) for learning purpose [34], [35], [64], [65]. Thus, the use/analysis of the SET data and information should be of paramount importance in management and heightening of the TEL-based education. For example, whereas the concept of ''datafication of learning'' which is also allied to analysis of the SET data done in this study, has brought about the emergence of the modern educational models and paved the way for new innovative methods for understanding and improvement of the teaching and learning processes [49], [66], [67]. We note, on the other hand, that the many educational institutions has to also take the extra steps of integrating the derived insights and information from analyzing those educational datasets (e.g. SET) into the curriculum as they are being developed, to be part of the teaching and learning processes, and help maintain/sustain the quality and idiosyncratical capacities of the educational models [1], [64], [68], [69], [70].

C. PEDAGOGICAL AND SOCIO-TECHNICAL FACTORS FOR TEL AND STUDENTS' EVALUATION OF TEACHING
With ''transformative education'' at the center of the many higher institutions' of learning goals [71], [72], [73], and ''technology'' spurring the educators to consume, innovate, and transfer knowledge and practices that transcends the stakeholders (e.g. teachers and students) into becoming global voices [74]. Our review of the current literature and state-of-the-art in education/learning delivery shows that the rigidity of the instructions or curriculum can present a threat to ample adoption and implementation of the TEL-based models for learning [75]. On the one hand, while the educational transformations that are experienced by the educators over the years, have led to substantial changes in the teaching and learning processes by adopting the digital technologies (augmented reality, online tutoring, and gamification elements, etc) [76] to foster online asynchronous simulation systems for teaching, and promotion of the learning processes including the wellbeing of students through the use of the technological tools [13], [76]. On the other hand, one of the transpiring challenges hand-in-hand with digital teaching (digitized-education) is the need for scalable strategies for development of the teaching-learning processes and practices. Moreover, the flexible digital models have shown its effectiveness in improving the learning outcomes, satisfaction, and progression for the students [13], [21], [77]. Interestingly, existing studies have shown that online feedback systems, such as the SET instrument described in this study, can be effective for understanding the different pathways or outcome (impact) of the teaching and learning processes, especially used for promotion of students' engagement and success [78].
To summarize the literatures, while the existing studies have shown that platforms such as m-learning or mobile applications have helped the students and educators in continuing education especially during the recent pandemic [18]. Likewise, in the past there are also studies that have fashioned digital programs to exclusively heighten the quality of teaching and learning for the students [79]. For example, the recent study that looked into pedagogical impact of the technologies within the classroom or educational settings [18], have shown that 53% of the participants agreed that technology has helped them greatly in their virtual learning/teaching process. While 87% disclosed that m-learning applications do motivate them for virtual learning, and 93% still believed that strengthening their digital skills in the use of virtual environments is also paramount to fully benefiting from the TEL-based education. In another study, Rio-Chillce et al. [80] found in a university setting that almost all the participants reported that they frequently use the digital platform for their online classes, with 68% agreeing that video conferencing platforms/tools help them to learn. 72% of the respondents also believed that they need to continue strengthening their digital knowledge, whereas less than 24% also stated that their academic performance has improved due to the new teaching modality or models (TEL-based) [80].
The comparative study of attitude, affect, motivation, cognitive engagement, and perceived behavioral control (ease of use, accessibility, self-efficacy) of students' use and acceptance of emergency online learning due to the recent global pandemic in USA, Mexico, Peru, and Turkey, shows that distance education or TEL-based education is continually experiencing transformation, and the students' cognitive engagement in the classroom settings has increased in many universities across the studied context [81]. Although, the study [81] equally mentioned how important it is to note that it was a challenge providing high-quality education to all students while continuing with the lockdowns or contingency measures during the pandemic, and most higher education institutions, globally, were not prepared with appropriate e-learning resources or online educational platforms to cater for the impromptu (virtual) mode of learning. It is also important to mention the fact that despite the many promising benefits of the TEL-based education and models, our review of the literature such as Suarez [59] noted that most institutions of learning have consequentially relaxed their assessment criteria so that students' academic performance is not affected, which can be used with repercussions or prone to ineffective learning outcomes or curricula. Indigenous communities or universities in the low-tech regions are also at a structural disadvantage as it concerns the TEL-based education [29], [30], [31], [82], as existing disparities and limited access to training, digital literacy and infrastructure between the rural vs urban regions has shown to be a socio-technical challenge or bottleneck to effective use and implementation of TEL-based models for learning [31], [82]. The UNESCO's Global Education Coalition [31] has also mentioned that while the TEL-based initiatives and technologies are promising, their underlying technicalities, educational goals, and services are only accessible to people with access to a computer and networks services, and in consequence, may breed educational ecosystems of social and technological inequality, or at the same time, span measures on how to scientifically determine and surrogate the pedagogical impact of the digitized-education on learning and outcomes, such as the one done in this study.

III. METHODOLOGY
TEL-based educational models [14], [15], [77] are learning approaches that integrate innovative teaching strategies and cutting-edge technologies, designed to improve the learning experiences of the teachers and students, and ensure continuous education through a combination of different components. As an example, the Flexible and Digital Model (MFD) [77] applied within a university setting is an educational model that consists of the amalgamation of interactive learning contents and activities, technological tools, and learning evaluation framework or assessment. Didactically, the main characteristics of such models (TEL-based) includes the flexibility to deliver learning or education at anywhere, any place, and at anytime, support of collaboration, monitoring, accompaniment, advice and feedback from the instructors, through remote interaction and tools, to availability of digital resources for active learning that are delivered using videos, web pages, canvas, and blackboard, web-conference sessions, and remote assisted work, etc [77]. While prior studies have looked into how best to apply the TEL-based models or initiatives to provide the teachers and students with productive and positive learning experiences [4], [5], [20], [55], [83], [84], [85], [86]. This study notes that the MFD educational model [77] comprises of several years' experience of the HEI in design and delivery of digital education programs aimed at ensuring academic continuity of more than 90 thousand students and 10 thousand instructors [77], [87] within the university setting.
In this study, we note that there is a need to explore pertinent methods for assessment and evaluation of the TEL-based approaches for learning [15]. In turn, while the use of digital (TEL) learning technologies and strategies has proved to be a way of enhancing the learning experiences for the students [4], [88], [89]. On the other hand, considering the students as consumers of the resultant educational models and technologies for learning, we note that there is also a need to monitor the impact or implications of the learning approaches and interventions on the students' performance.
To do this, this study applied a two-step (mixed) methodology using the text mining (qualitative) and statistical (quantitative) approach that is grounded on the data-structure and descriptive decision theory to analyze the dataset collected from the students' evaluation of teaching (SET) within the higher institutional setting to help determine and explain the pedagogical impact of the TEL-based models in education and/or learning outcomes and assessment.

A. DATA SAMPLING
The SET instrument designed for collection of information about the students' learning performance and outcomes, referred to as ECOA [90], is an institutional survey applied across 26 national campuses of the host institution where this research was conducted. The survey covers around 14 Schools and Divisions, 78 Departments, and 1082 Courses offered by the institution. For this study, we analyzed a total sample of n = 471968 comments provided by the undergraduate students in the survey between 2019 to 2021 during which the MFD model described in this study was implemented (see Methodology -Section III). The main construct and variables considered in the study include (i) the evaluation period between 2019 to 2021, and (ii) the type of evaluation (Students-to-Instructor, Student-to-Students, and overall Yes-No recommendation question) completed by the students.
In our experiment, we analyzed the three categories of SET data provided by the students (Students-to-Instructor, Student-to-Students, and overall Yes-No recommendation) in response to the following questions: i. Comments provided by the students about the instructors or their learning outcome (comentarios al instructor) ii. What would you say to a student who would like to register on the subject with the teacher? (qué le comentarías a un estudiante que quisiera inscribir la materia coneste profesor?) iii. Why would or wouldn't the student recommend the teacher? (por qué sí lo recomendarías or por qué no lo recomendarías?) From ethical point of view, we note that to obtain an unbiased analysis of the data, the names of the students who have completed the survey were withheld from the data, and the students were also informed about the purpose of the survey before completing the questionnaire, and were not directly involved in the analysis done in this study. Statistically, the study analyzed a total sample of n = 471968 comments after cleaning and removing the incomplete datasets or the students who did not comment in the data. We utilized the sample sizes of n 1 =390774 for the Students-to-Instructors' recommendation (REC), n 2 =66934 for the Students-to-Students REC, and n 3 =14260 for the Yes-No REC question, covering the period of 2019-2021, respectively.
Considering the reliability and validity of the analyzed dataset, we note that the ECOA instrument is an institutional survey administered and maintained by the host university, and has been used for several years by the institution for the purpose of evaluation of the teachers' performance based on answers provided by the students. The instrument has also been validated in previous studies [91], [90], [93]. The analyzed data (i.e., comments provided by the students) were a free choice open-ended question, and the estimated minimum sample size for this research purpose was 40 participants that we considered to be the scientifically acceptable size (n > 30 or 40) [94] for conducting the experimentations and data analysis when compared to the large enough sample size (n=471968) we have used. It is also important to note that the comments provided by the students were given and analyzed VOLUME 11, 2023 in Spanish. However, this study has reported and presented the results in English to cover the international audience and targeted readers of this paper.

B. EXPERIMENTAL SETUP
The authors developed a set of construct or research design it used to conduct its investigations as follows: • For the qualitative analysis, the study applied the educational process and data mining (EPDM) model (a text mining approach) to extract the different sentiments and emotional valence (textual data quantification or polarization) expressed by the students in the comments or SET instrument, and how those varies, if any, based on the SET type (Students-to-Instructor, Studentto-Students, and overall Yes-No recommendation) and period they have completed the evaluation (2019-2021).
• For the quantitative analysis, we conducted a multivariate analysis of co-variance (MANCOVA) to determine the effect or influence that the different type of SET and periods have on the average sentiment and emotions shown by the students, and how those may vary, if any, based on the significant factors by using a multiple pairwise comparisons post-hoc test.
• Finally, we evaluated the results of both the text mining and statistical analysis, and then provide an empirical discussion of the implications of the results for practice and/or adoption of the TEL-based educational initiatives or models across the HEIs.

IV. DATA ANALYSIS AND RESULTS
The data analysis were in two phases. In the first phase which consisted of the text mining analysis (qualitative approach) of the different comments provided by the students in the SET; we applied the EPDM model [52] to extract the average sentiment and emotional valence scores for the individual comments provided by the students between 2019 to 2021, broken down by the type of SET evaluation (i.e., students-instructors, students-students, yes-no REC, respectively). This was done to determine the main thresholds (polarization or intensities) or differences upon how the students may have rated their learning experiences and outcome during those periods (2019-2021).
In the second phase of our analysis, we conducted a multivariate analysis of co-variance (MANCOVA) to determine the marginal mean of effect that the evaluation periods (2019-2021) and SET types (students-instructors, studentsstudents, yes-no REC answer) have on the average sentiment and emotional valence scores we extracted from the comments (see Phase 1 description). Also, we used a multiple pairwise comparison post-hoc test to determine the main factors or where the significant differences may lie across the data. It is important to mention, in addition to considering the large sample size (n = 471968) for conducting the parametric test [94], that the study have applied the MANCOVA method considering the scale of measurement of the analyzed variables. Whereby the independent variable and co-variate (SET type and year) are represented as categorical variables with year considered as the particular interval of time or period (categorical), and the dependent variables (average sentiment and emotional valence scores) represented as continuous scale variables.
The results of both the qualitative (text mining) and quantitative (statistical) analysis are reported in detail in the following sections (Section -IV-A and IV-B), respectively.

A. QUALITATIVE ANALYSIS (TEXT MINING)
The text mining analysis (see Section III-B) was performed using R statistics tool [95]. The study used the sentiment_by, get_sentiments, and get_nrc_sentiment functions in R to extract the sentiment and emotional valence scores for the different comments provided by the students in the SET. Technically, the method focused on computing (through polarization or terms quantification) the average sentiments and intensities of the different emotions expressed by the students in the data by assigning a numerical score to each term/emotion found [52]. The outcome of the method (see: Tables 1 and 2) consists of quantified or polarized values that denote the intensities of the different terms (emotions) using the positive (+), neutral (0), and negative (-) connotations to represent each relevant term it finds in each iteration [41], [52], [96]. The resultant values with positive emotional valence (+) scores represents attractive emotions, whilst the negative (-) scores signify aversive emotions. The zeros (0) represent emotions that are classified as neutral by the model.
As gathered in Table 1, the average sentiment scores for the individual comments provided by the students in each matrix of the tables we built from the dataset shows that there were differences in the overall sentiment expressed by the students across the years (2019-2021), and they differ by SET evaluation type (Table 1). It is interesting to note that both the highest (max = 1.92695) and lowest (min = −1.18511) sentiment scores (see Table 1) was observed for the Students-Instructors recommendation, particularly for the period of 2019 in comparison to the following or subsequent years (2020, 2021). The study note that a number of factors may have led to this observation which are discussed in detail in the Discussion section (see Section V).
Furthermore, the authors considered it essential to determine the polarity or intensities (emotional valence) of the different combinations of words/comments provided by the students, and how those may vary based on the different types of emotions we found (see Figs. 2, 3, and 4). This was done based on the polarization method described in Okoye et al. [52]. To do this, we obtained the valence scores, which is computed by summing up the scores (polarity) of the words or terms the model identifies that can be used to express an emotion in the texts (comments), represented as either positive (++), neutral (0) or negative (-) values. The summary of the valence scores broken down by the SET type and period of evaluation (2019-2021) is reported in Table 2.  Table 2, there were differences in the overall intensities (emotional valence) of the comments provided by the students in the SET. It is noteworthy to mention that whilst the highest and lowest average sentiment (see :  Table 1) shown by the students was observed for the period of 2019 for the Students-instructor REC. In the emotional valence analysis (Table 2), we note that the most positive (max = 9.00) valence score was observed for the evaluation period of 2021 for the Students-instructors REC, with the lowest (negative valence) score (min = −5.00) reported for the Students-students REC for the period of 2020. Indeed, a number of factors may have led to the aforenoted observations, ranging from the uncertainties or anxieties that the students might have experienced as a result of the COVID-19 pandemic especially during its peak that happened in 2020, to factors that can be linked to migration issues from the routine face-to-face mode of learning to the remote learning mode or settings. However, the height of the emotions (max = 9.00) (see: Table 2) expressed by the students, that was observed for the period of 2021, could be as a result of how appreciative and reassured the students had eventually became following the technical and pedagogical support they received from their teachers and the institution, including as an indicator of the effectiveness of the MFD model that was implemented by the HEI to ensure the continuity of learning/teaching process for them during that time. Along these lines, the study turned its attention towards determining the different categories of emotions that the students have reported or expressed in the comments, and the differences that may exist amongst them based on the type of SET evaluation they have completed and periods (2019-2021). It is also important to mention that while the majority of the comments provided by the students in the SET were classified as neutral (i.e., values or valence = 0) (see: Fig. 1), thus, no emotional terms were found in the corresponding comments. The study reported in Figs. 2, 3, and 4, the different emotions' categories that it found for the students who did so.

As reported in
Emotional valence analysis (text mining method) and its implications in the different areas of its application particularly within the education domain, has been demonstrated in the literature [41], [52], [96], [97], [98], [99], [100]. As shown in this study, the EPDM model [52] was used to determine the polarity of the (textual) educational dataset (comments provided by the students in the SET). In Figs. 2, 3, and 4, the study applied the emotions' polarization or classifications of educational data as defined in Okoye et al. [52] and Litman & Forbes-Riley [96] to describe the different categories of emotions we found in the SET. This was done to establish the similarities and differences in the way the students have rated or expressed emotions in the comments, including its implications for instructional practice. It is noteworthy to mention that while the students trusts that their learning experiences were positive and are thus ''confident'' (approximately ∼25%) in the instructors and their learning outcome when making the recommendations (Fig. 2). They equally, on the other hand, expressed ''concern'' (∼20%) when making the recommendations to the other students (Fig. 3), including the yes-no recommendation (Figs. 4). However, considering the SET evaluation periods (2019-2021), we note that while there was no difference in the way the students have rated the instructors (Fig. 2 (A), (B), (C)) and the studentsstudents recommendation (Fig. 3 (A), (B), (C)). There was, on the other hand, a significant difference on how they have expressed emotions in the yes-no question or recommendation ( Fig. 4 (A), (B), (C)). With emotions such as ''resentful'' (see Fig. 4 (A), (2019)), ''concerned'' (Fig. 4 (B), (2020)), and ''uncertain'' (Fig. 4 (C), (2021)) coming out top, respectively. Also noteworthy is the fact that the students' recommendations or comments for the students-instructor construct ( Fig. 2 (A), (B), (C)) most significantly reflected ''confidence'' in the student' learning experiences or evaluation in comparison to the other SET types and emotions' categories we have found in the data.

B. QUANTITATIVE ANALYSIS (STATISTICAL METHOD)
In the quantitative analysis, the study turned its attention to determining the marginal means of effect or influence that the different evaluation periods (2019-2021) and SET type have on the extracted emotions and sentiment expressed by the students in the comments, and how these may differ, by making use of the quantified data (i.e., average sentiment and emotional valence scores) (see : Tables 1 and 2). To do this, we conducted a multivariate analysis of co-variance (MANCOVA) test to examine effect that the evaluation periods (2019-2021) and SET types have on the extracted average sentiment and emotional valence scores. The results of the statistical analysis are reported in Tables 3 and 4.
In the MANCOVA analysis reported in Table 3, we note by considering the ave_sentiment and emotional_valence scores, that while the SET evaluation_period and evaluation_type have a significant effect (p ≤.05) on the way the students  have rated the instructors (Students-to-Instructors) and the Students-to-Students REC (Table 3). On the other hand, they have not considered a combination of the both factors (evaluation_period * evaluation_type) when providing the comments, except for the emotional_valence that only came out significant for the Students-to-Students recommendation (F = 4.074, p =.044) ( Table 3). It is also noteworthy to mention that none of the factors (evaluation_period or  evaluation_type) had an effect on the sentiment or emotions expressed by the students for the REC yes-no question (Table 3).
Accordingly, the significant factors we have found specifically for the Students-to-Instructors and Students-to-Students RECs were tested using a multiple pairwise comparison posthoc test (adjusted with Bonferroni method) to determine where the significant differences lie across the data for the analyzed periods (2019-2021). The results are as shown in Table 4.

TABLE 3. Test of between-subjects effect for the ave_sentiment and emotional_valence broken down by the SET type and evaluation period (2019-2020).
As gathered in Table 4, the main differences were found between the period of 2019 and 2020 for the Students-Instructors REC evaluation in both the ave_sentiment (p =.030) and emotional_valence (p =.013), respectively. Whereas there was only significant difference in the emo-tional_valence (p =.045) for the Students-Student REC between the period of 2020 and 2021. As previously noted also in Table 3, no significant differences were found for the yes-no REC question (Table 4).

V. DISCUSSION
The pedagogical and technological transformations witnessed today especially in higher education has been linked to the use of digital technologies to enhance the teaching-learning processes [4], [9], [37], [43], [49], [64], [68], [101], [102], [103], [104], [105], [106]. However, those advancement comes alongside with several challenges that are anticipated to be addressed or taken into consideration by the different HEIs to gain the full benefit of the resultant TEL-based educational models and innovations. A conceptual understanding of the pedagogical impact of the new and emerging teaching models, for example the SET data analysis done in this study, can be one of the promising ways to help the educators improve the didactical instructions or strategies to drive the educational process forward.
Whereas, Engen [101] note that there is a need for a greater understanding of the new (educational) technologies and their effective use by the stakeholders (teachers and students). The works of Silva et al. [102] and Raffaghelli et al. [49], have also on the other hand, opined that the social, cultural, and technical aspects of the various users of the developed ''educational technologies'' toward the transformation and/or achievement of TEL-based education and initiatives must be considered, particularly as it concerns learning process management and outcomes, teaching spaces, planning, and innovative technologies that are supposedly used to support the students in their learning journey and performance [14], [15] In those perspectives, this study shows that by analyzing the educational datasets, e.g., SET, that the educators are not only able to get a better view or conceptual understanding of the technical structure or impact of the teaching/learning models and frameworks for the teachers and students, but can also extract useful information from the readily available (educational) data which can be used to inform or support the decision-making strategies and governance for the different institutions. Our approach through the integration of the text mining and statistical methods was carried out to uncover the perceptions/perspectives of the students who are deemed the direct consumers of the different educational models and initiatives using the EPDM model for textual data quantification and quantitative analysis, grounded on the Data-structure and Descriptive decision theory [50], [51]. This was done not only to understand what the students expect, or their learning needs could be, but also, to help identify what prominent factors that affects their learning experiences and/or how they view their learning outcomes and performances.
Indeed, while the study through the qualitative (text mining) and quantitative (statistical method) analysis or lens, VOLUME 11, 2023  found that there were significant differences in the average sentiment and emotions expressed by the students when completing the SET questionnaires (see: Figs. 2, 3, 4 and Tables 3 and 4). It also, on the other hand, found that the large margin of sentiment and emotions (i.e., positive and negative) shown by the students was observed for the Students-Instructor and Students-Students SET evaluations (Tables 1 and 2) as opposed to the Yes-No recommendation question that showed the least polarity (see Tables 1 and 2). This may mean that while the Students-Instructor and Student-Students SET questionnaire was an open-ended question therein the students had autonomy to provide their individual opinion about their learning experiences and outcome. On the other hand, the Yes-No REC question which was also an open-ended question but however was in response to the reason why they would or would not recommend the teachers, and therefore, may not provide them with the full autonomy or choice to freely express their feelings or emotions. Moreover, it is noteworthy to mention that while the students largely trust or were ''confident'' (∼25%) (Figs. 2, 3, and 4) in their learning experiences and outcomes, they also likewise expressed ''concern'' (∼20%) when making the recommendations, especially for the Students-Students and Yes-No REC questions (Figs. 2, 3, and 4).
Furthermore, in the text analysis (Table 2), where the study found for the evaluation periods, that the most positive emotional valence score (max = 9.00) was observed for the period of 2021, while the lowest (negative) score (min = −5.00) was found for the period of 2020. This observation can be explained to a number of factors: ranging from the uncertainty and anxieties the students may have developed as a result of the sudden shift to the remote mode of learning during the global pandemic at its peak in 2020, to technical challenges they also may have encountered upon migrating to the new (digital) learning platforms. Moreover, the top (max = 9.00) of emotions expressed by the students during the evaluation period of 2021 (Table 2) can be explained to how appreciative and reassured the students might have eventually became in relation to the different support (technical, didactical, and emotional wellbeing) they received during the unprecedented time of learning from their instructors and institution. Besides, this results could also be an indication of the positive impact or effectiveness of the flexible digital model (MFD) applied by the HEI at that time, not only to ensure the continuity of learning for the students, but also in ensuring that they are learning effectively with the right resources and in good condition.
As example, some specific comments provided by the students in the SET data in relation to both the ''positive'' and ''negative'' emotions the study have found include, respectively: ' Indeed, from the comments and observations, it can be said that ''flexibility'' of the offered courses or delivered contents was one of the top pedagogical provisions or cornerstones that the students found important in relation to their learning experiences and performance. Interestingly, HEIs are now integrating such features or element (flexibility) in the design of the several educational frameworks or learning platforms and curriculum. For instance, the MFD model [77], [87], [107] described in this paper, and the other hybrid models for learning [21], [108], [109] that didactically allows the students to choose and learn in either or both remote and in-person settings based on their learning needs and circumstances. Moreover, the flexible and digital models for learning is perceived to help tackle the problem of ''rigidity'' of the curriculums that has been identified as one of the main educational problems in both the literature and in practice [27], [30], [75], [82].
Accordingly, in the quantitative analysis (Section IV-A), therein we focused our attention on statistically determining the prominent factors that may have impacted or influenced the way the students have expressed the different sentiment and emotions in the data (see: Qualitative analysis -Section IV-B) by considering the evaluation periods (2019-2021) and SET types (Tables 3 and 4); it can be said that the SET evaluation_period and evaluation_type differed in terms of how the students have answered or rated the Students-to-Instructors and Students-to-Students questions (p ≤.05) (Table 3). Although, the result also showed that the students did not particularly take into account a combination of both constructs (i.e., evaluation_period * evaluation_type), except for the emotional_valence that came out significant for the Students-to-Students REC (p =.044) ( Table 3). In the multiple pairwise comparisons post-hoc test we further conducted to determine where the significant differences lie across the data (see Table 4), we found that the difference was observed for the period of 2019 and 2020 for the Students-Instructors recommendation in both the ave_sentiment (p =.030) and emotional_valence (p =.013) factors, respectively ( Table 4). The study only found significant differences in the emotional_valence (p =.045) for the Students-Student recommendation between the period of 2020 and 2021. Interestingly, the above results also triangulate with the results of the text mining analysis (qualitative approach) therein we have found that the most positive emotional valence score (max = 9.00) was for the evaluation period of 2021, while the lowest (negative) score (min = −5.00) was observed for the period of 2020 (Table 2), respectively.

A. IMPLICATIONS OF THIS STUDY
In education, while the pieces of evidence we drew from the literature and outcome of this study shows that the educators have invested in different innovative strategies and models to facilitate the teaching and learning processes for the teachers and students [8], [12], [18], [20], [64], [65], [74], [76], [81], [110], [111], [112]. The authors note that there still exists some issues and challenges in relation to the pedagogical transformations, hand-inhand with the impact of the digital teaching and acceleration of the new innovative models and practices form learning, that can only be achieved through a culture of educational innovation and conceptual understanding of how the effectiveness of the learning processes and outcomes are assessed [18], [76], [81], [113], [114]. A lot of higher educational institutions rely exclusively on SET evaluations for assessment of the teachers' performance and learning outcomes of the students [53], [115], [116], and there are also evidence in the literature that information (educational datasets) about the teaching and learning processes (which are stored at an unprecedented rate in the databases of the different institutions) can be used to understand and drive the educational systems forward [8], [27], [49], [89], [117], [118].
In triangulation with the method and objectives of this current study, educational organizations such as The Association for Institutional Research (AIR) [119] and The Professional and Organizational Development Network in Higher Education (POD) [120] are already pioneering several themes that are focused on students/learning assessment, institutional effectiveness and use of (educational) data for informed decision-making and support of the educators or educational processes to include (i) Assessment and Evaluation initiatives that are focused on research and methods that are both externally or internally driven from the institutional perspective for planning, accreditation, amongst others, in pursuit of improved and equitable student' and institutional success (ii) Data and Technology that features the technical and ethical creation/manipulation of data and structuring through appropriate analytic tools for decision making, predictive models, and machine learning (iii) Institutional Effectiveness which constitute of student-focused paradigm that explores the roles of information retrieval (IR), information extraction (IE), and Assessment in leading institutional data strategies and strategic planning for improvement of the students' success (iv) Use of Data that proves important for improved student success with attention to institutional and student contexts, [119], [120] etc., as uncovered in this study. Indeed, The POD [120] which supports change for the improvement of higher education through faculty, instructional, and organizational development has investigated the key issues of assessment and technology particularly for faculty development across the several institutions [121], and are pioneering discussions on new or ongoing educational, professional, or organizational development research that are systematically designed by employing wide-ranging methodologies and data analysis practices, such as the two step (mixed) method and analysis applied in this study (see Sections IV-A and IV-B. Consequentially, as one of the main contributions of this study is that it makes use of the SET data to analyze how the students perceive their learning performance and outcomes based on the comments they provided while completing the questionnaire. Our method which consisted of integration of the text mining and statistical approach, addresses one of the main themes or discourse in the current literature that suggests that the most pertinent way of determining the effectiveness of the teaching/learning processes can be combined with the students' comments to produce a more reliable and meaningful appraisal [122], [123]. Moreover, the need for ''data-driven'' segmentation or harvesting of data under technical archetypes, otherwise allied to the concept of ''datafication'' [45], [47], [124], [125], to help inform and improve the pedagogical practices of the higher institutions, as demonstrated in this study, has also been professed as one of the promising ways by which the higher education institutions can gain a better understanding or yet cross-sectional analysis of the learning needs of the students and their teachers [27], [41], [49], [82]. Although, Slade & Prinsloo [45] note that for the educators to gain an effective ''datafied-education'', the stakeholders (e.g., teachers and students) should also be part or take part in defining the context, purpose, and conditions under which or for which the collected educational datasets (e.g., SET) are used and implemented. Besides, an ample analysis and conceptual understanding of the SET can offer context-appropriate solutions for effective design of the educational curriculum, models or frameworks [45], and can provide new and better ways to track, monitor, and improve the educational ecosystems at large [7], [8], [27], [49], [82], [126].
In the wider spectrum of scientific research and global education practice, particularly as it concerns the TEL-based education, the authors note that while the digital technologies should be part of any innovative strategies by the educators, and to attain an effective online teaching and selfefficacy [1], [114], [127]. On the other hand, it is important to mention the fact that these also comes with the portentous task of ensuring that the users (teachers and students) are imbued with the much-needed digital competencies, and adequate or even alternative solutions to replacing the in-person learning environments that suppositionally comprises a sizeable part of the present-day educational models and curriculum [61], [128], [129], [130], [131], [132]. The idiosyncratic or strategic tasks and action plans by the educators should also include guaranteeing proper and secured virtual learning environments or platforms for the stakeholders, to provision of uninterrupted access to the internet and network infrastructures [127], [133]. Also noteworthy is the fact that the United Nations Educational, Scientific and Cultural Organization (UNESCO) under its Global Education Coalition (GEC) initiative [111] have stipulated that never before have the educational system witnessed disruption at a global scale, and partnership amongst the concerned stakeholders is one of the only ways forward. They [111] have called for ''coordinated'' and ''innovative'' actions that are aimed at unlocking solutions that supports the teachers and their students with the teaching/learning process, and these also include methods that aim to utilize (educational) information or data stored about the users, such as the one done in this study, to inform and improve the educational process [42], [111], [134].
The recent studies that have also looked into how the transitions to the remote learning or TEL-based education have affected the students [61], [129], [135], have also discovered that both their academic and technological needs have proportionally increased, especially following the impact of crises such as the recent Covid-19 pandemic in education [61], [126], [135], [136], [137]. Some of the main challenges that have emerged as a result of the so-called transitions include: that it will be difficult to replicate the face-to-face learning experiences for the students online [130], [132] notwithstanding that the remote learning can be as good or better than in-person learning for those or students who choose it [138].
Pedagogical -wise, concerning the complementarity of the new and emerging educational technologies or TEL-based education, by comparing the synchronous and asynchronous learning modes, it can be said that while most of the students have shown to prefer ''asynchronous'' mode of learning due to its flexibility, and the fact that they can learn at their own pace, at any time and any place [18], [81], [129], [139]. On the other hand, the students that preferred ''synchronous'' teaching style also indicated that it motivated and kept them up-to-date with learning [18], [81], [129], [139]. Moreover, it is also worth mentioning that many of the higher institutions of learning are now implementing the hybrid educational models (virtual and in-person) for teaching and learning purposes and outcomes, by allowing the students to choose the best mode of learning suitable for their learning needs and circumstances [13], [21], [23], [140].

B. LIMITATIONS AND OPPORTUNITIES FOR FUTURE RESEARCH
While this study has introduced the text mining (qualitative) and statistical (quantitative) method of analysis for a conceptualized outlook on the prominent factors that impacts or influences the learning experiences and performances by the students, the authors acknowledges that the study could also come with some limitations. For example, although the study has used the assimilation of the data-structure approach and descriptive decision theory to study the rationale behind the decisions that the leaners are disposed to make, there could be potentially many other ways to approach this, particularly within the education domain, or other methods that may have not been considered yet in this study. For instance, the process of identification of the learning foundations or components that can be used to aptly enable flexibility in the different learning platforms or tools, or ways to evaluate the interoperability or technical functionalities of the resultant platforms based on the individual learning settings or contexts can form another direction for future research. Also, the textual data quantification and its use for other further types of analysis such as the statistical (quantitative) analysis done in this paper, stands as a new innovation or methodological approach to analyzing the educational datasets and domain, and to the best of our knowledge, there are no studies except for the authors', currently in the literature, that have expounded on this method. Therefore, the authors note that this study represents as a methodological road map or incentives to more robust research to come particularly as it concerns the assessment of the impact of SET in education. Moreover, future studies can adopt the method described in this paper to analyze data about any given educational process or domain, or yet, transfer -ability of the method to include other components or analysis that may have not already been done in this paper.

VI. CONCLUSION
This study used SET data (n = 471968) collected within a higher education setting to analyze how the students evaluated the teachers and their learning performance. Qualitatively, it applied the EPDM model (a text mining approach) to extract the different sentiment and emotional valence shown by the students when completing the SET questionnaires.
Consequently, the study quantitatively analyzed the extracted information (average sentiment and emotional valence) using the MANCOVA and multiple comparisons tests to determine the marginal mean of effect that the different SET types and evaluation periods (2019-2021) have on the students' performance or views about the teaching-learning process. The pedagogical implications of the results and key findings for effective TEL-based education and practice was also discussed. Whilst the scholastic indicator from the study shows that the flexible digital models or instructional methods are effective for continuous education, innovative pedagogies, and teaching transformations. It also, on the other hand, serve as an incentive for more robust research to come that aim to explore its main implications for students' learning outcome and assessment. For instance, the results of this study should be helpful in designing questionnaires for students' satisfaction concerning the teachers' teaching, and improving of the didactical instructions or strategies. Future research can be further extended to this aspect.
KINGSLEY OKOYE (Member, IEEE) received the B.Sc. degree in computer science, in 2007, the M.Sc. degree in technology management, in 2011, and the Ph.D. degree in software engineering from the University of East London, U.K., in 2017. He worked as a Software Programmer and Development Tutor for the undergraduates. He has also worked as a Data Architect. He is currently a Research Professor (Mentor) with the Center for Educational Innovation, Institute for Future of Education, Tecnológico de Monterrey. He is also a devoted Researcher in both industry and academia in hardware and software fields of computing, such as data science, machine learning, artificial intelligence, big data and advanced analytics, software development and programming, and business process management. His mission is to foster sustainable technical research and development, and provide solutions through critical thinking, creative problem-solving, and cross-functional collaboration. He has contributed to research and project outcomes by participating, assessing, and evaluating their impact upon the scientific and industrial communities. He has more than 35 scientific publications published as books, journal articles, book chapters, and conference proceedings (including the Best Paper Award) in high-indexed Q1/Q2 reputable journals, book publishers, and conferences, in the areas of computing, higher education, and educational innovation, as a result of his research. His research interests include process mining and automation, semantic web technologies, learning analytics and systems design, computing education, educational innovation, educational technologies, knowledge engineering and data management, internet applications, and ontology. He is a member of the Institution of Engineering and Technology (IET), U.K., the Mexican National Academy of Researchers ( He is also a Leader who has worked for the last 12 years in management and decisionmaking positions. The first two years were spent in the private sector, and the next ten years in the education sector at the Tecnológico de Monterrey. His experience relies on leadership, strategic planning, and high-performance team (HPT) creation focused on achieving results based on a strategy of forming HPTs in a favorable working environment, which allows the organization to differentiate in the short and medium term. Professionally, his strengths and motivation are focused on right team's conformation and the personal growth of each of its members, and the development and implementation of strategies for a sustained competitive advantage within the organization.