ACADEMIA Letters Learning Analytics: Framing the right question for the right data to impact teaching and learning effectiveness Fay Patel Abstract This article focuses on a rapidly developing area of data analytics which is to gather and utilize learning analytics to support teaching inquiry with the intention of improving teaching and learning effectiveness. The author asserts that in order to collect the right data, it is imperative that academics, administrators, leadership teams and institutional data analytics staff frame the right question. The right question will glean the right data which in turn will provide the most effective strategies to improve a relevant component of teaching and learning practice. Keywords: learning analytics, data analytics, teaching and learning inquiry Introduction Various approaches to the gathering and usage of learning analytics (LA) are evident glocally (on a local and global scale). The emphasis to date has been in gathering institutional data in general. Producing relevant data analytics for various international higher education rankings, student enrolment, and graduation rates has been a common objective. Data analytics has for decades attracted criticism in regard to the ethical considerations, violations of privacy, and biases in the use and application of learning analytics in general (Gupta and Saxena, 2021; Cerrato Pargman & McGrath; 2021). This article acknowledges the wider debate as noted by Academia Letters, August 2021 ©2021 by the author — Open Access — Distributed under CC BY 4.0 Corresponding Author: Fay Patel, dr.fay.patel@gmail.com Citation: Patel, F. (2021). Learning Analytics: Framing the right question for the right data to impact teaching and learning effectiveness. Academia Letters, Article 2974. https://doi.org/10.20935/AL2974. 1 Gupta and Saxena. However, it is focused specifically on the positive components of learning analytics which impact improvements in teaching and learning practices with its critical role in enhancing the education quality of the teaching and learning experience. In particular, the article brings attention to the importance of framing the right question to retrieve “the right data at the right time” in teaching and learning inquiry. The author utilizes the term “learning analytics” as a general term for all descriptions of teaching and learning related analytics which have the purpose of improving and impacting the enhancement of teaching and learning practices. The RMIT University (an Australian higher education institution) presents one of the most articulate information briefs on learning analytics in a transparent manner on their website. The opening statement that learning analytics “act as custodians of RMIT’s data for education, who gather, analyze and present data to enhance the learning and teaching experience” places in context the rationale and the governance policy framework of data analytics at the institution. The RMIT team (2021) have also identified the five core work areas (facilitative, descriptive, diagnostic, prescriptive and predictive), which direct the framing of the right question aligning it with the purpose for which it will be utilized, together with eight principles to safeguard “integrity and transparency“ of their data analysis process. The five core areas provide guidelines in the framing of the right question. Further, the RMIT Learning Analytics website makes an expressive statement about the important linkage of learning analytics to the enhancement of learning and teaching practice. The university places emphasis on evidence -based decisions “with the right data at the right time”. This article expands on that approach to emphasize that framing the right question with purpose will produce the right data at the right time. This will guarantee that the retrieved learning analytics will make a meaningful contribution to the strategic improvement of teaching and learning effectiveness. Gathering the right data (learning analytics) requires purpose (why it is required, what is the justification for gathering a specific learning data set, and how it will be applied to meet the objective of improving specific teaching and learning practice). Therefore, framing the question for teaching and learning inquiry is the most important step in the learning analytics gathering process. The question has to be clearly articulated so that the academic and the data analyst are in agreement about what is required and how it is to be utilized to improve teaching and learning effectiveness. This is an important stage of the learning analytics journey. It is the scaffold between the specialist disciplinary knowledge base of the academic (educator) and the specialist learning analytics team member (data analyst) who retrieves and examines the data. Sergis and Sampson (2017, p.2) contend that there are barriers to framing the teaching inquiry (right question) such as the “data literacy” competencies of the educator. Other barriers may include the limitation of the data analyst to Academia Letters, August 2021 ©2021 by the author — Open Access — Distributed under CC BY 4.0 Corresponding Author: Fay Patel, dr.fay.patel@gmail.com Citation: Patel, F. (2021). Learning Analytics: Framing the right question for the right data to impact teaching and learning effectiveness. Academia Letters, Article 2974. https://doi.org/10.20935/AL2974. 2 interpret the framed question within the academic disciplinary context, and/or to interpret it within the broader context of the language of teaching and learning effectiveness. Teaching Analytics, Student Analytics, Learning Analytics In the glocal (local and global) literature, teaching and learning inquiry has led to a number of different labels for data analytics that is specifically targeted at improving and enhancing teaching and learning impact in the glocal classroom. Sergis and Samson (2017, p.2) maintain that different strands of data analytics are emerging to define the boundary lines among the barriers however in different regions of the world, different labels are used to describe data analytics that are specifically aligned to the improvement of teaching and learning practices. Often, teaching, student and learning analytics are used interchangeably (as in this article) in the literature to distinguish teaching and learning oriented data analysis from the general categories of data analysis that is focused on institutional rankings, graduation rates and enrolment trends. Although Sergis and Sampson dichotomize teaching analytics as reference to teacher’s actions in education design and delivery, and learning analytics as reference to learner’s actions, the author does not subscribe to such a distinction. Global Trends in Learning Analytics The literature (Gupta & Saxena, 2021; Rufallo Noel Levitz, 202; Ndukwe & Daniel, 2020; Sergis & Sampson, 2017; Deloitte, 2016) offers a wide range of perspectives and approaches to embedding learning analytics as a teaching and learning improvement supportive measure. A cursory glimpse into the learning analytics trends around the world regions (from the author’s experience and perspective) suggests that except for a few international higher education institutions which adopted approaches that have successfully impacted education quality, other approaches lacked vision, careful planning and understanding of the connectivity between learning analytics and teaching and learning enhancements. In Australia, RMIT University (2021) has demonstrated an in-depth and insightful approach to the improvement of higher education across their international footprint; in previous years (2014-2015) the transnational campus (of the Australian) Monash University in Malaysia implemented a unique and sustained quality enhancement approach to review the end of term scores across disciplines to identify “subjects at risk” instead of students at risk and/or educators at risk. In an effort to enhance “subjects at risk”, the institution implemented a curriculum improvement plan for the next term, adopted and successfully implemented the Peer Assisted Student Sessions (PASS) program (Patel et al, 2017), formally known as the Academia Letters, August 2021 ©2021 by the author — Open Access — Distributed under CC BY 4.0 Corresponding Author: Fay Patel, dr.fay.patel@gmail.com Citation: Patel, F. (2021). Learning Analytics: Framing the right question for the right data to impact teaching and learning effectiveness. Academia Letters, Article 2974. https://doi.org/10.20935/AL2974. 3 Supplemental Instruction program in the United States in earlier decades, and which was focused on raising the quality of “subjects at risk.” In one of the Hong Kong institutions (2018), learning analytics was expected to be aligned with quality education improvements. It was presented in two parts: first, the data analyst savvy educational development staff presented the data; next, the educational development lead led a discussion about the impact the data might have on teaching and learning enhancement. However, there was an absence of a discussion about the alignment of the data analysis to a specific area of teaching and learning improvement. In Canada, the University of Regina in Saskatchewan was the first university (in the authors travels through North America, Australasia and the Asian Pacific) to adopt a Teaching and Student Analytics portfolio as a centralized campus wide initiative in 2019 (President’s Report, 2019, p.2). In all the aforementioned efforts, the “data literacy competency” barriers mentioned earlier (Sergis and Sampson, 2017), the lack of a set of guidelines to connect specific learning analytics to a particular component of teaching and learning improvement, the absence of framing the right questions to gather the “right data at the right time” together with the dichotomization of teaching and learning enhancement from the student analytics portfolio created various challenges. Conclusion Gupta and Saxena (2021) raise pertinent issues in regard to the need for “robust debate” around the subject of the “ethics of learning analytics”. It is time to combine that debate with critical questions about how to retrieve relevant and useful data aligned to the right questions to generate critical teaching and learning inquiry (El Khaber, Moulay & Cherif, 20 18; Sergis & Sampson, 2017) which will propel the enhancement of the quality of teaching and learning across international higher education. There is no doubt that “LA allows educators to plan timely and targeted educational interventions based on learner habits”; and that advocates “who are pushing for greater use of LA in the domain of education and training” (Gupta and Saxena (2021, p.2) have a plan of action to align relevant learning analytics with specific improvements in teaching and learning practice. References Cerratto Pargman, T., & McGrath, C. (2021). Mapping the Ethics of Learning Analytics in Higher Education: A Systematic Literature Review of Empirical Research. Journal of Learning Analytics, 1-17. https://doi.org/10.18608/jla.2021.1 Academia Letters, August 2021 ©2021 by the author — Open Access — Distributed under CC BY 4.0 Corresponding Author: Fay Patel, dr.fay.patel@gmail.com Citation: Patel, F. (2021). Learning Analytics: Framing the right question for the right data to impact teaching and learning effectiveness. Academia Letters, Article 2974. https://doi.org/10.20935/AL2974. 4 Deloitte. (2016). Student Analytics Enabling personal, proactive and fact-based student services https://www2.deloitte.com/content/dam/Deloitte/nl/Documents/deloitte-analytics/deloittenl-data-analyse-student-analytics-fact-based-student-services.pdf Gupta, A., Saxena, N. (2021). Need for a robust debate around the ethics of Learning Analytics. Academia Letters, Article 907. https://doi.org/10.20935/AL907 El-Khaber, H., Moulay, H.A.H., & Cherif, Z. (2018). Teaching and Learning Analytics to Support Teacher Inquiry International Journal of Engineering and Technology Vol 7, No 4.32 (2018)(Special Issue 32):44-47 Ndukwe, I.G., & Daniel, B.K. (2020) Teaching analytics, value and tools for teacher data literacy: a systematic and tripartite approach. Int J Educ Technol High Educ 17, 22 (2020). https://doi.org/10.1186/s41239-020-00201-6 Patel, Fay & Saipul, Fadhliyansah & Chan, Regina. (2017). Enhancing the 21st Century Learning Experience:. 10.4018/978-1-5225-1689-7.ch011. RMIT University. (2021). University Website. https://www.rmit.edu.au/ RMIT. (2021). Learning Analytics. https://www.rmit.edu.au/about/governance-management/ structure/education/learning-analytics Ruffalo Noel & Levitz (2021). Student Success Maximize College Student Retention and Completion College Student Retention | College Student Graduation Rates | Ruffalo Noel Levitz (ruffalonl.com) Sergis S., & Sampson D.G. (2017) Teaching and Learning Analytics to Support Teacher Inquiry: A Systematic Literature Review. In: Peña-Ayala A. (eds) Learning Analytics: Fundaments, Applications, and Trends. Studies in Systems, Decision and Control, vol 94. Springer, Cham. https://doi.org/10.1007/978-3-319-52977-6_2 University of Regina. (2019). President’s Report to the Board September 2019 p.2. https:// www.uregina.ca/president/assets/docs/pdf/USec/September-2019-Board-Report.pdf Academia Letters, August 2021 ©2021 by the author — Open Access — Distributed under CC BY 4.0 Corresponding Author: Fay Patel, dr.fay.patel@gmail.com Citation: Patel, F. (2021). Learning Analytics: Framing the right question for the right data to impact teaching and learning effectiveness. Academia Letters, Article 2974. https://doi.org/10.20935/AL2974. 5