Jelena Jovanović
ABSTRACT
Models and processes developed in learning analytics research are increasing in sophistication and predictive power. However, the ability to translate analytic findings to practice remains problematic. This study aims to address this issue by establishing a model of learner behaviour that is both predictive of student course performance, and easily interpreted by instructors. To achieve this aim, we analysed fine grained trace data (from 3 offerings of an undergraduate online course, N=1068) to establish a comprehensive set of behaviour indicators aligned with the course design. The identified behaviour patterns, which we refer to as observed study strategies, proved to be associated with the student course performance. By examining the observed strategies of high and low performers throughout the course, we identified prototypical pathways associated with course success and failure. The proposed model and approach offers valuable insights for the provision of process-oriented feedback early in the course, and thus can aid learners in developing their capacity to succeed online.
- Baker, R.S. 2016. Stupid Tutoring Systems, Intelligent Humans. International Journal of Artificial Intelligence in Education. (Feb. 2016), 1--15.Google Scholar
- Bjork, R.A., Dunlosky, J. and Kornell, N. 2013. Self-Regulated Learning: Beliefs, Techniques, and Illusions. Annual Review of Psychology, 64, 1, 417--444.Google ScholarCross Ref
- Boroujeni, M.S. and Dillenbourg, P. 2018. Discovery and Temporal Analysis of Latent Study Patterns in MOOC Interaction Sequences. Proceedings of the 8th International Conference on Learning Analytics and Knowledge (New York, NY, USA, 2018), 206--215.Google ScholarDigital Library
- Butler, D.L. and Winne, P.H. 1995. Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65, 3, 245--281.Google ScholarCross Ref
- Cerezo, R., Bogarín, A., Esteban, M. and Romero, C. 2019. Process mining for self-regulated learning assessment in e-learning. Journal of Computing in Higher Education. Google ScholarCross Ref
- Doran, D., Schulz, S. and Besold, T.R. 2017. What Does Explainable AI Really Mean? A New Conceptualization of Perspectives. arXiv:1710.00794 [cs].Google Scholar
- Ferreira, D., Zacarias, M., Malheiros, M. and Ferreira, P. 2007. Approaching Process Mining with Sequence Clustering: Experiments and Findings. Business Process Management (Berlin, Heidelberg, 2007), 360--374.Google Scholar
- Fincham, O.E., Gasevic, D.V., Jovanovic, J.M. and Pardo, A. 2018. From Study Tactics to Learning Strategies: An Analytical Method for Extracting Interpretable Representations. IEEE Transactions on Learning Technologies. Google ScholarCross Ref
- Gašević, D., Dawson, S. and Siemens, G. 2014. Let's not forget: Learning analytics are about learning. TechTrends. 59, 1 (Dec. 2014), 64--71.Google Scholar
- Gasevic, D., Jovanovic, J., Pardo, A. and Dawson, S. 2017. Detecting Learning Strategies with Analytics: Links with Self-reported Measures and Academic Performance. Journal of Learning Analytics, 4, 2 (Jul. 2017), 113--128.Google ScholarCross Ref
- Gatta, R., Lenkowicz, J., Vallati, M., Rojas, E., Damiani, A., Sacchi, L., Bari, B.D., Dagliati, A., Fernandez-Llatas, C., Montesi, M., Marchetti, A., Castellano, M. and Valentini, V. 2017. pMineR: An Innovative R Library for Performing Process Mining in Medicine. Artificial Intelligence in Medicine, 351--355.Google Scholar
- Greene, J.A., Plumley, R.D., Urban, C.J., Bernacki, M.L., Gates, K.M., Hogan, K.A., Demetriou, C. and Panter, A.T. 2019. Modeling temporal self-regulatory processing in a higher education biology course. Learning and Instruction. Google ScholarCross Ref
- Hadwin, A.F., Nesbit, J.C., Jamieson-Noel, D., Code, J. and Winne, P.H. 2007. Examining trace data to explore self-regulated learning. Metacognition and Learning, 2, 107--124.Google ScholarCross Ref
- Hattie, J. and Timperley, H. 2007. The power of feedback. Review of Educational Research, 77, 1, 81--112.Google ScholarCross Ref
- Herodotou, C., Rienties, B., Boroowa, A., Zdrahal, Z. and Hlosta, M. 2019. A large-scale implementation of predictive learning analytics in higher education: the teachers' role and perspective. Educational Technology Research and Development, 67, 5, 1273--1306.Google ScholarCross Ref
- Jamieson-Noel, D. and Winne, P.H. 2003. Comparing Self-Reports to Traces of Studying Behavior as Representations of Students' Studying and Achievement. German Journal of Educational Psychology, 17, 3-4, 159--171.Google Scholar
- Jovanović, J., Gašević, D., Dawson, S., Pardo, A. and Mirriahi, N. 2017. Learning analytics to unveil learning strategies in a flipped classroom. The Internet and Higher Education, 33, 74--85.Google ScholarCross Ref
- Jovanovic, J., Mirriahi, N., Gašević, D., Dawson, S. and Pardo, A. 2019. Predictive power of regularity of pre-class activities in a flipped classroom. Computers & Education, 134, 156--168.Google ScholarDigital Library
- Kizilcec, R.F., Pérez-Sanagustín, M. and Maldonado, J.J. 2017. Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses. Computers & Education, 104, (Jan. 2017), 18--33.Google Scholar
- Lim, L.-A., Gentili, S., Pardo, A., Kovanović, V., Whitelock-Wainwright, A., Gašević, D. and Dawson, S. 2019. What changes, and for whom? A study of the impact of learning analytics-based process feedback in a large course. Learning and Instruction. Google ScholarCross Ref
- Liu, D.Y.-T., Bartimote-Aufflick, K., Pardo, A. and Bridgeman, A.J. 2017. Data-Driven Personalization of Student Learning Support in Higher Education. Learning Analytics: Fundaments, Applications, and Trends: A View of the Current State of the Art to Enhance e-Learning. A. Peña-Ayala, ed. Springer International Publishing. 143--169.Google Scholar
- Lowes, S., Lin, P. and Kinghorn, B. 2015. Exploring the Link between Online Behaviours and Course Performance in Asynchronous Online High School Courses. Journal of Learning Analytics, 2, 2 (Dec. 2015), 169--194.Google ScholarCross Ref
- Lust, G., Elen, J. and Clarebout, G. 2013. Regulation of tool-use within a blended course: Student differences and performance effects. Computers & Education, 60, 1, 385--395.Google ScholarDigital Library
- Maldonado-Mahauad, J., Pérez-Sanagustín, M., Kizilcec, R.F., Morales, N. and Munoz-Gama, J. 2018. Mining theory-based patterns from Big data: Identifying self-regulated learning strategies in Massive Open Online Courses. Computers in Human Behavior, 80, (Mar. 2018), 179--196.Google Scholar
- Matcha, W., Gašević, D., Uzir, N.A., Jovanović, J. and Pardo, A. 2019. Analytics of Learning Strategies: Associations with Academic Performance and Feedback. Proceedings of the 9th International Conference on Learning Analytics & Knowledge (New York, NY, USA, 2019), 461--470.Google ScholarDigital Library
- Matcha, W., Uzir, N.A., Gasevic, D. and Pardo, A. 2019. A Systematic Review of Empirical Studies on Learning Analytics Dashboards: A Self-Regulated Learning Perspective. IEEE Transactions on Learning Technologies. Google ScholarDigital Library
- McCabe, J. 2011. Metacognitive awareness of learning strategies in undergraduates. Memory & Cognition, 39, 3 (Apr. 2011), 462--476.Google ScholarCross Ref
- Panzarasa, P., Kujawski, B., Hammond, E.J. and Roberts, C.M. 2016. Temporal patterns and dynamics of e-learning usage in medical education. Educational Technology Research and Development, 64, 1 (Feb. 2016), 13--35.Google ScholarCross Ref
- Pardo, A., Bartimote, K., Shum, S.B., Dawson, S., Gao, J., Gašević, D., Leichtweis, S., Liu, D., Martínez-Maldonado, R., Mirriahi, N., Moskal, A.C.M., Schulte, J., Siemens, G. and Vigentini, L. 2018. OnTask: Delivering Data-Informed, Personalized Learning Support Actions. Journal of Learning Analytics, 5, 3, 235--249.Google ScholarCross Ref
- Pardo, A., Gašević, D., Jovanovic, J.M., Dawson, S. and Mirriahi, N. 2018. Exploring Student Interactions with Preparation Activities in a Flipped Classroom Experience. IEEE Transactions on Learning Technologies. Google ScholarDigital Library
- Rienties, B., Herodotou, C., Olney, T., Schencks, M. and Boroowa, A. 2018. Making Sense of Learning Analytics Dashboards: A Technology Acceptance Perspective of 95 Teachers. International Review of Research in Open and Distributed Learning, 19, 5. Google ScholarCross Ref
- Rosé, C.P., McLaughlin, E.A., Liu, R. and Koedinger, K.R. 2019. Explanatory learner models: Why machine learning (alone) is not the answer. British Journal of Educational Technology, 50, 2943--2958. Google ScholarCross Ref
- Schmitz, M., Scheffel, M., van Limbeek, E., Bemelmans, R. and Drachsler, H. 2018. "Make It Personal!" - Gathering Input from Stakeholders for a Learning Analytics-Supported Learning Design Tool. Lifelong Technology-Enhanced Learning (Cham, 2018), 297--310.Google ScholarDigital Library
- Schwendimann, B.A., Rodríguez-Triana, M.J., Vozniuk, A., Prieto, L.P., Boroujeni, M.S., Holzer, A., Gillet, D. and Dillenbourg, P. 2017. Perceiving Learning at a Glance: A Systematic Literature Review of Learning Dashboard Research. IEEE Transactions on Learning Technologies, 10, 1, 30--41.Google ScholarDigital Library
- Winne, P.H. 2013. Learning Strategies, Study Skills, and Self-Regulated Learning in Postsecondary Education. Higher Education: Handbook of Theory and Research. M.B. Paulsen, ed. Springer Netherlands. 377--403.Google Scholar
- Winne, P.H. 2017. Leveraging big data to help each learner and accelerate learning science. Teachers College Record, 119, 3, 1--24.Google Scholar
- Zhou, M. and Winne, P.H. 2012. Modeling academic achievement by self-reported versus traced goal orientation. Learning and Instruction, 22, 6, 413--419.Google ScholarCross Ref
Index Terms
- Supporting actionable intelligence: reframing the analysis of observed study strategies
Recommendations
The importance and meaning of session behaviour in a MOOC
AbstractOne of the main challenges for online learners is knowing how to effectively manage their time. Highly autonomous settings, such as Massive Open Online Courses (MOOCs), put additional pressure on learners in this regard. However, ...
Highlights- We used learning analytics to examine learners' session behaviour patterns in a MOOC.
Supporting learners' self-regulated learning in Massive Open Online Courses
AbstractIn MOOCs, learners are typically presented with great autonomy over their learning process. Therefore, learners should engage in self-regulated learning (SRL) in order to successfully study in a MOOC. Learners however often struggle to self-...
Highlights- Learners struggle to regulate their learning in massive open online courses (MOOCs).
- A self-regulated learning (SRL) intervention was implemented in three MOOCs.
- Learners' SRL was measured with trace data variables.
- ...
Examining change in students’ self-regulated learning patterns after a formative assessment using process mining techniques
AbstractThe growing popularity of e-learning platforms, such as learning management systems, has foregrounded the role of self-regulated learning (SRL) in student success. In many e-learning environments, students typically complete learning assignments ...
Highlights- Students engage in a loosely sequenced recursive self-regulated learning cycle.
- Students actively adjust self-regulated learning patterns in response to a formative assessment.
- There are group differences in the adjustment of self-...
Comments