Skip to main content
SpringerLink
Log in

Enhancing learner experience with instructor cues in video lectures: A comprehensive exploration and design discovery toward a novel gaze visualization

  • Published:
Education and Information Technologies Aims and scope Submit manuscript

Abstract

Pre-recorded lectures have become a prevalent approach in online education due to the proliferation of MOOC platforms and the COVID-19 pandemic. However, due to the lack of real-time interactions between instructors and learners, learners have encountered various difficulties in understanding the lectures and actively engaging with the learning materials. Therefore, various attempts have been made to aid learning by revealing the cues of the instructor, including countenance, digital pointer, and gaze, in video lectures. Nevertheless, the current body of research still lacks a comprehensive exploration of these non-verbal cues, critical for understanding and enhancing the learning experience. Consequently, recent studies on visualization even with novel cues often fail to match the learners’ needs. Therefore, in this paper, we explore the impact of instructors’ non-verbal cues, focusing on the countenance, digital pointer, and gaze, in video lectures and introduce a novel gaze visualization called Elements-aware Gaze. First, we determine learners’ design preferences and optimize design parameters for Elements-aware Gaze. Afterward, we analyze the usability, workload, and cognitive load of non-verbal cues in video lectures through a user study with 37 subjects. The findings highlight the practicality of the Digital Pointer and the effectiveness of Elements-aware Gaze, recognized for its utility and engaging learning experience while minimizing the workload in video lectures. Finally, our results emphasize the importance of thoughtful design and strategic integration of non-verbal cues in video lectures, providing a valuable foundation for enhancing remote learning experiences in the evolving online education landscape.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability Statement

The datasets analyzed during the current study are available from the corresponding author upon reasonable request.

References

  • Alpizar, D., Adesope, O. O., & Wong, R. M. (2020). A meta-analysis of signaling principle in multimedia learning environments. Educational Technology Research and Development, 68, 2095–2119.

    Article  Google Scholar 

  • Aydin, I. E., & Yazici, M. (2020). Drop-out in moocs. Turkish Online Journal of Educational Technology-TOJET, 19(3), 9–17.

    Google Scholar 

  • Bao, W. (2020). Covid-19 and online teaching in higher education: A case study of peking university. Human Behavior and Emerging Technologies, 2(2), 113–115.

    Article  Google Scholar 

  • Barfield, W., & Furness, T. A. (1995). Virtual environments and advanced interface design. Oxford University Press.

    Book  Google Scholar 

  • Bennett, G.K., Seashore, H.G., & Wesman, A.G. (1947). Differential aptitude tests.

  • Berger, V. W., & Zhou, Y. (2014). Kolmogorov-smirnov test: Overview. Wiley statsref: Statistics reference online.

    Google Scholar 

  • Bhat, S., Chinprutthiwong, P., & Perry, M. (2015). Seeing the instructor in two video styles: Preferences and patterns. International Educational Data Mining Society.

  • Blascheck, T., Kurzhals, K., Raschke, M., & et al. (2014). State-of-the-art of visualization for eye tracking data. In: Eurovis (stars) (p. 29).

  • Bojović, Ž, Bojović, P. D., Vujošević, D., amp, et al. (2020). Education in times of crisis: Rapid transition to distance learning. Computer Applications in Engineering Education, 28(6), 1467–1489.

  • Chen, C. M., & Wu, C. H. (2015). Effects of different video lecture types on sustained attention, emotion, cognitive load, and learning performance. Computers & Education, 80, 108–121.

    Article  Google Scholar 

  • D’angelo, S., & Schneider, B. (2021). Shared gaze visualizations in collaborative interactions: past, present and future. Interacting with Computers, 33(2), 115–133.

    Article  Google Scholar 

  • Dumford, A. D., & Miller, A. L. (2018). Online learning in higher education: exploring advantages and disadvantages for engagement. Journal of Computing in Higher Education, 30, 452–465.

    Article  Google Scholar 

  • Emhardt, S. N., Jarodzka, H., Brand-Gruwel, S., et al. (2022). What is my teacher talking about? effects of displaying the teacher’s gaze and mouse cursor cues in video lectures on students’ learning. Journal of Cognitive Psychology, 34(7), 846–864.

    Article  Google Scholar 

  • Gal, Y., & et al. (2016). Uncertainty in deep learning.

  • Guo, P.J., Kim, J., & Rubin, R. (2014). How video production affects student engagement: An empirical study of mooc videos. In: Proceedings of the first ACM conference on Learning@ scale conference (pp. 41–50).

  • Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (task load index): Results of empirical and theoretical research. Advances in Psychology (pp. 139–183). Advances in psychology: Elsevier.

    Google Scholar 

  • Henderson, M. L., & Schroeder, N. L. (2021). A systematic review of instructor presence in instructional videos: Effects on learning and affect. Computers and Education Open, 2(100), 059.

    Google Scholar 

  • Hinds, P. J. (1999). The curse of expertise: The effects of expertise and debiasing methods on prediction of novice performance. Journal of Experimental Psychology: Applied, 5(2), 205.

    Google Scholar 

  • Hirzle, T., Sauter, M., Wagner, T., & et al. (2022). Attention of many observers visualized by eye movements. In: 2022 Symposium on eye tracking research and applications (pp. 1–7).

  • Kim, D., Jeong, J.Y., Hong, S., & et al. (2022). Visualizing instructor’s gaze information for online video-based learning: Preliminary study. In: 2022 Symposium on Eye Tracking Research and Applications (pp. 1–6).

  • Klepsch, M., Schmitz, F., & Seufert, T. (2017). Development and validation of two instruments measuring intrinsic, extraneous, and germane cognitive load. Frontiers in psychology, 8, 1997.

    Article  Google Scholar 

  • Lee, D.W., Ahuja, C., Liang, P.P., & et al. (2022). Multimodal lecture presentations dataset: Understanding multimodality in educational slides. arXiv:2208.08080

  • Li, I., Fabbri, A.R., Tung, R.R., & et al. (2019). What should i learn first: Introducing lecturebank for nlp education and prerequisite chain learning. In: Proceedings of the AAAI conference on artificial intelligence (pp. 6674–6681).

  • Mayer, R.E., & Pilegard, C. (2005). Principles for managing essential processing in multimedia learning: Segmenting, pretraining, and modality principles. The Cambridge handbook of multimedia learning (pp. 169–182).

  • Mayer, R. E., & Moreno, R. (1998). A cognitive theory of multimedia learning: Implications for design principles. Journal of educational psychology, 91(2), 358–368.

    Google Scholar 

  • Mayer, R. E., & Moreno, R. E. (2010). Techniques that reduce extraneous cognitive load and manage intrinsic cognitive load during multimedia learning. Cambridge University Press.

    Book  Google Scholar 

  • Ocepek, U., Bosnić, Z., Šerbec, I. N., et al. (2013). Exploring the relation between learning style models and preferred multimedia types. Computers & Education, 69, 343–355.

    Article  Google Scholar 

  • Oh, J., & Jeong, J.W. (2023). Deep learning-based layout detection in online lectures. Proceedings of the korean HCI society (pp. 1156–1159).

  • Oliveira, G., Grenha Teixeira, J., Torres, A., et al. (2021). An exploratory study on the emergency remote education experience of higher education students and teachers during the covid-19 pandemic. British Journal of Educational Technology, 52(4), 1357–1376.

    Article  Google Scholar 

  • Pfitzmann, B., Auer, C., Dolfi, M., & et al. (2022). Doclaynet: A large human-annotated dataset for document-layout segmentation. In: Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining (pp. 3743–3751).

  • Pi, Z., Deng, L., Wang, X., et al. (2022). The influences of a virtual instructor’s voice and appearance on learning from video lectures. Journal of Computer Assisted Learning, 38(6), 1703–1713.

  • Pi, Z., Liu, W., Ling, H., et al. (2023). Does an instructor’s facial expressions override their body gestures in video lectures? Computers & Education, 193(104), 679.

    Google Scholar 

  • Pi, Z., Xu, K., Liu, C., et al. (2020). Instructor presence in video lectures: Eye gaze matters, but not body orientation. Computers & Education, 144(103), 713.

  • Polat, H. (2022). Instructors’ presence in instructional videos: A systematic review. Education and information technologies (pp. 1–33).

  • Revilla-Cuesta, V., Skaf, M., Varona, J. M., et al. (2021). The outbreak of the covid-19 pandemic and its social impact on education: Were engineering teachers ready to teach online? International Journal of Environmental Research and Public Health, 18(4), 2127.

  • Richter, J., Scheiter, K., & Eitel, A. (2016). Signaling text-picture relations in multimedia learning: A comprehensive meta-analysis. Educational Research Review, 17, 19–36.

    Article  Google Scholar 

  • Rosch, J. L., & Vogel-Walcutt, J. J. (2013). A review of eye-tracking applications as tools for training. Cognition, technology & work, 15, 313–327.

    Article  Google Scholar 

  • Rosenthal, S., & Walker, Z. (2020). Experiencing live composite video lectures: Comparisons with traditional lectures and common video lecture methods. International Journal for the Scholarship of Teaching and Learning, 14(1), 8.

    Article  Google Scholar 

  • Sauter, M., Wagner, T., & Huckauf, A. (2022). Distance between gaze and laser pointer predicts performance in video-based e-learning independent of the presence of an on-screen instructor. In. (2022). Symposium on Eye Tracking Research and Applications. New York, NY, USA: ACM.

  • Sauter, M., Wagner, T., Hirzle, T., et al. (2023). Behind the screens: Exploring eye movement visualization to optimize online teaching and learning. Mensch und Computer, 2023, 67–80.

  • Schrepp, M., Hinderks, A., & Thomaschewski, J. (2017). Design and evaluation of a short version of the user experience questionnaire (ueq-s). International Journal of Interactive Multimedia and Artificial Intelligence, 4(6), 103–108.

    Article  Google Scholar 

  • Semmelmann, K., & Weigelt, S. (2018). Online webcam-based eye tracking in cognitive science: A first look. Behavior Research Methods, 50, 451–465.

    Article  Google Scholar 

  • Sharma, K., Jermann, P., & Dillenbourg, P. (2015). Displaying teacher’s gaze in a mooc: Effects on students’ video navigation patterns. In: Design for teaching and learning in a networked world: 10th European conference on technology enhanced learning, EC-TEL 2015, Toledo, Spain, September 15-18, 2015, Proceedings 10 (pp. 325–338). Springer.

  • Sorden, S. D. (2012). The cognitive theory of multimedia learning. Handbook of educational theories, 1(2012), 1–22.

    Google Scholar 

  • Špakov, O., Niehorster, D., Istance, H., & et al. (2019). Two-way gaze sharing in remote teaching. Human-Computer Interaction - INTERACT 2019 (pp. 242–251). Lecture notes in computer science: Springer International Publishing, Cham.

  • Stull, A. T., Fiorella, L., & Mayer, R. E. (2018). An eye-tracking analysis of instructor presence in video lectures. Computers in Human Behavior, 88, 263–272.

    Article  Google Scholar 

  • Sweller, J., van Merriënboer, J. J., & Paas, F. (2019). Cognitive architecture and instructional design: 20 years later. Educational psychology review, 31, 261–292.

    Article  Google Scholar 

  • Tulaskar, R., & Turunen, M. (2022). What students want? experiences, challenges, and engagement during emergency remote learning amidst covid-19 crisis. Education and Information Technologies, 27(1), 551–587.

    Article  Google Scholar 

  • Vagg, T., Balta, J. Y., Bolger, A., et al. (2020). Multimedia in education: what do the students think? Health Professions Education, 6(3), 325–333.

    Article  Google Scholar 

  • Van Gog, T. (2014). 11–the signaling (or cueing) principle in multimedia learning. The Cambridge handbook of multimedia learning (pp. 263–278).

  • Vandenberg, S. G., & Kuse, A. R. (1978). Mental rotations, a group test of three-dimensional spatial visualization. Perceptual and motor skills, 47(2), 599–604.

    Article  Google Scholar 

  • Wagner, T., Hirzle, T., Huckauf, A., & et al (2023) Exploring gesture and gaze proxies to communicate instructor’s nonverbal cues in lecture videos. In: Extended Abstracts of the 2023 CHI conference on human factors in computing systems, pp 1–7.

  • Wang, C.Y., Bochkovskiy, A., & Liao, H.Y.M. (2023). Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 7464–7475).

  • Wang, J., & Antonenko, P. D. (2017). Instructor presence in instructional video: Effects on visual attention, recall, and perceived learning. Computers in human behavior, 71, 79–89.

    Article  Google Scholar 

  • Wang, J., Antonenko, P., & Dawson, K. (2020). Does visual attention to the instructor in online video affect learning and learner perceptions? an eye-tracking analysis. Computers & Education, 146(103), 779.

    Google Scholar 

  • Wang, H., Pi, Z., & Hu, W. (2019). The instructor’s gaze guidance in video lectures improves learning. Journal of Computer Assisted Learning, 35(1), 42–50.

    Article  Google Scholar 

  • Wilson, K. E., Martinez, M., Mills, C., et al. (2018). Instructor presence effect: Liking does not always lead to learning. Computers & Education, 122, 205–220.

  • Yao, N., Brewer, J., D’Angelo, S., & et al. (2018). Visualizing gaze information from multiple students to support remote instruction. In: Extended abstracts of the 2018 CHI conference on human factors in computing systems (pp. 1–6).

  • Zhang, S., de Koning, B.B., & Paas, F. (2023). Effects of finger and mouse pointing on learning from online split-attention examples. British Journal of Educational Psychology.

  • Zhang, Z., & Zhang, J. (2010). A strong tracking nonlinear robust filter for eye tracking. Journal of Control Theory and Applications, 8, 503–508.

    Article  MathSciNet  Google Scholar 

  • Zhang, Z., & Zhang, J. (2010). A new real-time eye tracking based on nonlinear unscented kalman filter for monitoring driver fatigue. Journal of Control Theory and Applications, 8, 181–188.

    Article  Google Scholar 

Download references

Funding

This study was financially supported by Seoul National University of Science and Technology.

Author information

Authors and Affiliations

  1. Seoul National University of Science and Technology, Seoul, South Korea

    Jae-Yeop Jeong, JiYeon Oh & Jin-Woo Jeong

Authors
  1. Jae-Yeop Jeong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. JiYeon Oh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jin-Woo Jeong
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: [JYJ, JWJ]; Methodology: [JYJ, JWJ]; Software: [JYJ, JYO, JWJ]; Validation: [JYJ, JYO, JWJ]. Formal analysis: [JYJ, JWJ]. Investigation: [JYJ, JWJ]. Resources: [JYJ, JYO, JWJ]. Data curation: [JYJ, JYO, JWJ]. Writing-original draft preparation: [JYJ, JYO, JWJ]. Writing-review and editing: [JYJ, JWJ]. Visualization, [JYJ]. Supervision: [JWJ]. Project administration: [JWJ]. Funding acquisition: [JWJ]. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Jin-Woo Jeong.

Ethics declarations

Competing interests

The authors declare that they have no conflict of interest

Informed consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jeong, JY., Oh, J. & Jeong, JW. Enhancing learner experience with instructor cues in video lectures: A comprehensive exploration and design discovery toward a novel gaze visualization. Educ Inf Technol (2024). https://doi.org/10.1007/s10639-024-12697-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10639-024-12697-w

Keywords

Search

Navigation