Abstract
Lower student success rates in higher education might, in some case, be due to the unprecedented increase of student numbers without a comparable increase in resources and funding. This paper proposes a face-based detection system to monitor occupancy and student attention in crowded classroom using state of the art deep Convolutional Neural Networks (CNN) architectures. The aim of the proposed system is to contribute to the increase of subject success rates by monitoring attendance and attention. The system utilizes a two-phased approach: The first phase determines the number of student faces in an image frame. The Haar Cascade, LBP, HOG, Resnet CNN, TinyFace CNN, and SSD were compared to determine the algorithm best suited to the detection of faces in crowded classroom scenes. In phase two, the orientations of the faces are determined using transfer learning. Faces are classified as “right”, “left”, or at the “center”. This information is displayed on an augmented reality display to provide feedback to lecturers in semi real-time. It is hoped that this will assist lecturers to address problems related to student attention in crowded classrooms.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Monks, J., Schmidt, R.: The Impact of Class Size and Number of Students on Outcomes in Higher Education. Robins School of Business, University of Richmond. https://www.ilr.cornell.edu/sites/ilr.cornell.edu/files/WP136.pdf. Accessed July 2019
Ciresan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: CVPR (2012)
Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Deepface: closing the gap to human-level performance in face verification. In: CVPR (2014)
Sun, Y., Chen, Y., Wang, X., Tang, X.: Deep learning face representation by joint identification-verification. In: NIPS (2014)
Wan, L., Zeiler, M., Zhang, S., Cun, Y.L., Fergus, R.: Regularization of neural networks using dropconnect. In: ICML, pp. 1058–1066 (2013)
Dodge, S., Karam, L.: A study and comparison of human and deep learning recognition performance under visual distortions. arXiv:1705.02498v1 [cs.CV], May 2017
Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Conference on Computer Vision and Pattern Recognition (2001)
Kushsairy, K., et al.: A comparative study between LBP and Haar-like features for face detection using OpenCV, pp. 335–339 (2014). https://doi.org/10.1109/ice2t.2014.7006273
Rekha, N., Kurian, M.Z.: Face detection in real time based on HOG. Int. J. Adv. Res. Comput. Eng. Technol. (IJARCET) 3(4), 1345–1352 (2014)
OpenCV. https://www.opencv.org/. Accessed July 2019
DLib. http://blog.dlib.net/. Accessed July 2019
Hu, P., Ramanan, D.: Finding Tiny Faces. Robotics Institute, Carnegie Mellon. arXiv:1612.04402v2 [cs.CV], April 2017
Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2. https://github.com/weiliu89/caffe/blob/ssd/README.md
Krizhevsky, A.: ImageNet Classification with Deep Convolutional Neural Networks. http://www.image-net.org/challenges/LSVRC/2012/supervision.pdf. Accessed July 2019
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017). https://doi.org/10.1145/3065386. ISSN 0001-0782
Simonyan, K., Zisserman, A.: Very Deep Convolutional Networks For Large – Scale Image Recogntion. arXiv:1409.1556v6 [cs.CV], April 2015
He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition. arXiv:1512.03385 [cs.CV], December 2015
LeCun, Y., et al.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Szegedy, C., et al.: Going deeper with convolutions. arXiv:1409.4842v1 [cs.CV], September 2014
Szegedy, C., et al.: Rethinking Inception Architecture for Computer Vision. arXiv:1512.00567v3 [cs.CV], December 2015
Szegedy, C., et al.: Inception-v4, Inception-ResNet and the impact of Risudual Connections on Learning. arXiv:1602.07261v2 [cs.CV], August 2016
Huang, G., Liu, Z., Weinberger, K.Q., van der Maaten, L.: Densely Connected Convolutional Networks. arXiv:1608.06993 [cs.CV], August 2016
Peer, P., et al.: Strategies for exploiting independent cloud implementations of biometric experts in multibiometric scenarios. Math. Probl. Eng. 2014, 1–15 (2014)
Gourier, N., Hall, D., Crowley, J.L.: Estimating face orientation from robust detection of salient facial features. In: Proceedings of Pointing, ICPR, International Workshop on Visual Observation of Deictic Gestures, Cambridge, UK (2014)
Acknowledgements
The face images used in this work have been provided by the Computer Vision Laboratory, University of Ljubljana, Slovenia [1, 2].
The training face images from CVL Face Database used in this work have been provided by the Computer Vision Laboratory, University of Ljubljana, Slovenia.
The test face images from Pointing Face Database used in this work has been provided by the ICPR, International Workshop on Visual Observation of Deictic Gestures, Cambridge, UK.
The merSETA chair in Intelligent Manufacturing at TUT is thanked for its financial support.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Smith, A.J., Van Wyk, B.J., Du, S. (2019). CNNs and Transfer Learning for Lecture Venue Occupancy and Student Attention Monitoring. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2019. Lecture Notes in Computer Science(), vol 11845. Springer, Cham. https://doi.org/10.1007/978-3-030-33723-0_31
Download citation
DOI: https://doi.org/10.1007/978-3-030-33723-0_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-33722-3
Online ISBN: 978-3-030-33723-0
eBook Packages: Computer ScienceComputer Science (R0)