Abstract
The study of student attention is an important topic in education because this type of analysis provides important information to teachers to potentially improve the quality of their classes. In this paper, we present AATiENDe, a system that uses emotion recognition, gaze direction approximation and body posture analysis as features to classify whether students are paying attention to their computer screens. To do this, we use a mixture of deep learning-based techniques and novel machine learning techniques applied to tabular classifiers to produce the final predictions. We also capture and label a customized dataset to train the models. Our approach provides over 90% accuracy using two cameras and over 80% accuracy using only the foreground camera.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bosch, N.: Detecting student engagement: human versus machine. In: Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization, UMAP ’16, pp. 317–320. Association for Computing Machinery, New York, NY, USA (2016). https://doi.org/10.1145/2930238.2930371
Barbadekar, A., et al.: Engagement index for classroom lecture using computer vision. In: Global Conference for Advancement in Technology (GCAT), 2019, pp. 1–5 (2019)
Canedo, D., Trifan, A., Neves, A.J.R.: Monitoring students’ attention in a classroom through computer vision. In: Bajo, J., et al. (eds.) PAAMS 2018. CCIS, vol. 887, pp. 371–378. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-94779-2_32
Li, W., Jiang, F., Shen, R.: Sleep gesture detection in classroom monitor system. In: ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7640–7644 (2019)
Dinesh, A.N.S.D., Bijlani, K.: Student analytics for productive teaching/learning. In: 2016 International Conference on Information Science (ICIS), pp. 97–102 (2016)
Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vision 57(2), 137–154 (2004)
Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking-learning-detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1409–1422 (2011)
Martin, S., Tran, C., Trivedi, M.: Optical flow based head movement and gesture analyzer (ohmega). In: Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012), pp. 605–608. IEEE (2012)
Liao, W., Xu, W., Kong, S., Ahmad, F., Liu, W.: A two-stage method for hand-raising gesture recognition in classroom, pp. 38–44 (2019). https://doi.org/10.1145/3318396.3318437
Selim, T., Elkabani, I., Abdou, M.A.: Students engagement level detection in online e-learning using hybrid efficientnetb7 together with TCN, LSTM, and Bi-LSTM. IEEE Access 10, 99:573–99:583 (2022)
Tan, M., Le, Q.: Efficientnet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114. PMLR (2019)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Sig. Process. 45(11), 2673–2681 (1997)
Bai, S., Kolter, J.Z., Koltun, V.: An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018)
Christian Mejia-Escobar, E.M.-M., Cazorla, M.: Towards a better performance in facial expression recognition: a data-centric approach. In: Computational Intelligence and Neuroscience (2023)
Bhallaakshit: Facial expression recognition, September 2020. https://www.kaggle.com/code/bhallaakshit/facial-expression-recognition/notebook
Suwarno, S., Kevin, K.: Analysis of face recognition algorithm: Dlib and OpenCV. J. Inform. Telecommun. Eng. 4(1), 173–184 (2020)
Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1867–1874 (2014)
Valentin Bazarevsky, E.G.B., Grishchenko, I.: Blazepose: on-device real-time body pose tracking. In: CVPR Workshop on Computer Vision for Augmented and Virtual Reality. ACM, August 2020. https://doi.org/10.1145/2F2939672.2939785
Lugaresi, C., et al.: Mediapipe: a framework for building perception pipelines (2019). https://arxiv.org/abs/1906.08172
Chen, T., Guestrin, C.: XGBoost. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, August 2016. https://doi.org/10.1145/2F2939672.2939785
Ke, G., et al.: LightGBM: a highly efficient gradient boosting decision tree. Adv. Neural. Inf. Process. Syst. 30, 3146–3154 (2017)
Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. Roy. Stat. Soc. Ser. B (Stat. Methodol.) 67(2), 301–320 (2005)
McCulloch, W.S., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5(4), 115–133 (1943)
Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)
Delvigne, V., Wannous, H., Dutoit, T., Ris, L., Vandeborre, J.-P.: Phydaa: physiological dataset assessing attention. IEEE Trans. Circuits Syst. Video Technol. 32(5), 2612–2623 (2022)
Chong, E., Wang, Y., Ruiz, N., Rehg, J.M.: Detecting attended visual targets in video. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2020
Fan, S., et al.: Emotional attention: a study of image sentiment and visual attention. In:. IEEE/CVF Conference on Computer Vision and Pattern Recognition 2018, pp. 7521–7531 (2018)
Acknowledgments
This work has been carried out under the framework of the grant CIPROM/2021/17 funded by Prometeo program from Conselleria de Innovación, Universidades, Ciencia y Sociedad Digital of Generalitat Valenciana (Spain). This work has also been funded by a PhD grant under the reference UAFPU21-78 from the University of Alicante (Spain).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Escalona, F., Gomez-Donoso, F., Morillas-Espejo, F., Pina-Navarro, M., Marquez-Carpintero, L., Cazorla, M. (2023). AATiENDe: Automatic ATtention Evaluation on a Non-invasive Device. In: Rojas, I., Joya, G., Catala, A. (eds) Advances in Computational Intelligence. IWANN 2023. Lecture Notes in Computer Science, vol 14135. Springer, Cham. https://doi.org/10.1007/978-3-031-43078-7_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-43078-7_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43077-0
Online ISBN: 978-3-031-43078-7
eBook Packages: Computer ScienceComputer Science (R0)