Abstract
In learning situations that do not occur exclusively online, the analysis of multimodal evidence can help multiple stakeholders to better understand the learning process and the environment where it occurs. However, Multimodal Learning Analytics (MMLA) solutions are often not directly applicable outside the specific data gathering setup and conditions they were developed for. This paper focuses specifically on authentic situations where MMLA solutions are used by multiple stakeholders (e.g., teachers and researchers). In this paper, we propose an architecture to process multimodal evidence of learning taking into account the situation’s contextual information. Our adapter-based architecture supports the preparation, organisation, and fusion of multimodal evidence, and is designed to be reusable in different learning situations. Moreover, to structure and organise such contextual information, a data model is proposed. Finally, to evaluate the architecture and the data model, we apply them to four authentic learning situations where multimodal learning data was collected collaboratively by teachers and researchers.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
This is not an UML-based data model but it is a sample data model.
- 2.
Observational form available at http://tiny.cc/adek5y.
- 3.
Reference implementation source code available at http://tiny.cc/7oek5y.
- 4.
Example Graasp log available at http://tiny.cc/avek5y.
- 5.
Example observation data available at http://tiny.cc/1wek5y.
- 6.
A sample of the configuration file is available at http://tiny.cc/ead95y.
References
Bano, M., Zowghi, D.: User involvement in software development and system success: a systematic literature review. In: Proceedings of the 17th International Conference on Evaluation and Assessment in Software Engineering, pp. 125–130. ACM (2013)
Buschmann, F., Henney, K., Schmidt, D.C.: Pattern-Oriented Software Architecture, on Patterns and Pattern Languages, vol. 5. Wiley, Hoboken (2007)
Chatti, M.A., et al.: Learning analytics: challenges and future research directions. eleed 10(1) (2014)
Chen, L., Feng, G., Leong, C.W., Joe, J., Kitchen, C., Lee, C.M.: Designing an automated assessment of public speaking skills using multimodal cues. J. Learn. Anal. 3(2), 261–281 (2016)
Di Mitri, D., Schneider, J., Klemke, R., Specht, M., Drachsler, H.: Read between the lines: an annotation tool for multimodal data for learning. In: Proceedings of the 9th International Conference on Learning Analytics & Knowledge, pp. 51–60. ACM (2019)
Farrell, T., Mikroyannidis, A., Alani, H.: “We’re seeking relevance”: qualitative perspectives on the impact of learning analytics on teaching and learning. In: Lavoué, É., Drachsler, H., Verbert, K., Broisin, J., Pérez-Sanagustín, M. (eds.) EC-TEL 2017. LNCS, vol. 10474, pp. 397–402. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66610-5_33
Grafsgaard, J., Wiggins, J., Boyer, K.E., Wiebe, E., Lester, J.: Predicting learning and affect from multimodal data streams in task-oriented tutorial dialogue. In: Proceedings of the 7th International Conference on Educational Data Mining. Citeseer (2014)
Hernández-García, Á., Conde, M.Á.: Dealing with complexity: educational data and tools for learning analytics. In: Proceedings of the Second International Conference on Technological Ecosystems for Enhancing Multiculturality, pp. 263–268. ACM (2014)
Krafzig, D., Banke, K., Slama, D.: Enterprise SOA: Service-Oriented Architecture Best Practices. Prentice Hall Professional, Upper Saddle River (2005)
Miller, H.G., Mork, P.: From data to decisions: a value chain for big data. IT Prof. 15(1), 57–59 (2013). https://doi.org/10.1109/MITP.2013.11
Ochoa, X.: Multimodal learning analytics. In: Handbook of Learning Analytics, pp. 129–141 (2017)
Ochoa, X., Worsley, M.: Augmenting learning analytics with multimodal sensory data. J. Learn. Anal. 3(2), 213–219 (2016)
Prieto, L.P., Rodríguez-Triana, M.J., Martínez-Maldonado, R., Dimitriadis, Y., Gašević, D.: Orchestrating learning analytics (OrLA): supporting inter-stakeholder communication about adoption of learning analytics at the classroom level. Australas. J. Educ. Technol. 4(35) (2019). https://doi.org/10.14742/ajet.4314
Rodríguez-Triana, M.J., Prieto, L.P., Martínez-Monés, A., Asensio-Pérez, J.I., Dimitriadis, Y.: The teacher in the loop: customizing multimodal learning analytics for blended learning. In: Proceedings of the 8th International Conference on Learning Analytics and Knowledge, pp. 417–426. ACM (2018)
Rodriguez Triana, M.J., et al.: Monitoring, awareness and reflection in blended technology enhanced learning: a systematic review. Int. J. Technol. Enhanced Learn. 9(2–3), 126–150 (2017)
Schneider, J., Börner, D., Van Rosmalen, P., Specht, M.: Augmenting the senses: a review on sensor-based learning support. Sensors 15(2), 4097–4133 (2015)
Schneider, J., Di Mitri, D., Limbu, B., Drachsler, H.: Multimodal learning hub: a tool for capturing customizable multimodal learning experiences. In: Pammer-Schindler, V., Pérez-Sanagustín, M., Drachsler, H., Elferink, R., Scheffel, M. (eds.) EC-TEL 2018. LNCS, vol. 11082, pp. 45–58. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-98572-5_4
Shankar, S.K., Prieto, L.P., Rodríguez-Triana, M.J., Ruiz-Calleja, A.: A review of multimodal learning analytics architectures. In: 2018 IEEE 18th International Conference on Advanced Learning Technologies (ICALT), pp. 212–214. IEEE (2018)
Tarr, P., Ossher, H., Harrison, W., Sutton, S.M.: N degrees of separation: multi-dimensional separation of concerns. In: Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No. 99CB37002), pp. 107–119. IEEE (1999)
Worsley, M.: Multimodal learning analytics as a tool for bridging learning theory and complex learning behaviors. In: Proceedings of the 2014 ACM Workshop on Multimodal Learning Analytics Workshop and Grand Challenge, pp. 1–4. ACM (2014)
Worsley, M., Blikstein, P.: Leveraging multimodal learning analytics to differentiate student learning strategies. In: Proceedings of the Fifth International Conference on Learning Analytics and Knowledge, pp. 360–367. ACM (2015)
Acknowledgements
This research has been partially funded by the European Union via the European Regional Development Fund and in the context of CEITER and Next-Lab (Horizon 2020 Research and Innovation Programme, grant agreements no. 669074 and 731685).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Shankar, S.K., Ruiz-Calleja, A., Prieto, L.P., Rodríguez-Triana, M.J., Chejara, P. (2019). An Architecture and Data Model to Process Multimodal Evidence of Learning. In: Herzog, M., Kubincová, Z., Han, P., Temperini, M. (eds) Advances in Web-Based Learning – ICWL 2019. ICWL 2019. Lecture Notes in Computer Science(), vol 11841. Springer, Cham. https://doi.org/10.1007/978-3-030-35758-0_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-35758-0_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-35757-3
Online ISBN: 978-3-030-35758-0
eBook Packages: Computer ScienceComputer Science (R0)