Elena Di Lascio
Skip Abstract Section
Abstract
Modern wearable devices enable the continuous and unobtrusive monitoring of human physiological parameters, including heart rate and electrodermal activity. Through the definition of adequate models these parameters allow to infer the wellbeing, empathy, or engagement of humans in different contexts. In this paper, we show that off-the-shelf wearable devices can be used to unobtrusively monitor the emotional engagement of students during lectures. We propose the use of several novel features to capture students' momentary engagement and use existing methods to characterize the general arousal of students and their physiological synchrony with the teacher. To evaluate our method we collect a data set that -- after data cleaning -- contains data from 24 students, 9 teachers, and 41 lectures. Our results show that non-engaged students can be identified with high reliability. Using a Support Vector Machine, for instance, we achieve a recall of 81% -- which is a 25 percentage points improvement with respect to a Biased Random classifier. Overall, our findings may inform the design of systems that allow students to self-monitor their engagement and act upon the obtained feedback. Teachers could profit of information about non-engaged students too to perform self-reflection and to devise and evaluate methods to (re-)engage students.
- Mary Ainley. 2012. Students' Interest and Engagement in Classroom Activities. In Handbook of Research on Student Engagement. Springer.Google Scholar
- Douglas G Altman. 1990. Practical Statistics for Medical Research. CRC Press.Google Scholar
- Kimberly E Arnold, Brandon Karcher, Casey V Wright, and James McKay. 2017. Student Empowerment, Awareness, and Self-Regulation Through a Quantified-Self Student Tool. In Proceedings of the 7th ACM International Learning Analytics 8 Knowledge Conference (LAK 2017). Google ScholarDigital Library
- Ivon Arroyo, David G. Cooper, Winslow Burleson, Beverly Park Woolf, Kasia Muldner, and Robert Christopherson. 2009. Emotion Sensors Go To School. In Proceedings of the Conference on Artificial Intelligence in Education: Building Learning Systems That Care: From Knowledge Representation to Affective Modelling (AIED 2009). Google ScholarDigital Library
- Jorn Bakker, Mykola Pechenizkiy, and Natalia Sidorova. 2011. What's Your Current Stress Level? Detection of Stress Patterns from GSR Sensor Data. In Proceedings of the IEEE International Conference on Data Mining Workshops. Google ScholarDigital Library
- Christian Beckel, Leyna Sadamori, Thorsten Staake, and Silvia Santini. 2014. Revealing Household Characteristics from Smart Meter Data. Energy.Google Scholar
- Mathias Benedek and Christian Kaernbach. 2010. Decomposition of Skin Conductance Data By Means of Nonnegative Deconvolution. Psychophysiology.Google Scholar
- Christopher M Bishop. 2006. Pattern Recognition and Machine Learning. Springer. Google ScholarDigital Library
- Brandon Booth, Asem Ali, Ian Bennett, and Shrikanth Narayanan. 2017. Toward Active and Unobtrusive Engagement Assessment of Distance Learners (ACII 2017). In Proceedings of 7th International Conference on Affective Computing and Intelligent Interaction.Google Scholar
- Wolfram Boucsein. 2012. Electrodermal Activity. Springer Science 8 Business Media.Google Scholar
- Saša Branković. 2012. Assessment of Brain Monoaminergic Signaling Through Mathematical Modeling of Skin Conductance Response. In Neuroscience-Dealing With Frontiers. InTech.Google Scholar
- John T Cacioppo, Louis G Tassinary, and Gary Berntson. 2007. Handbook of psychophysiology. Cambridge University Press.Google Scholar
- Ryan Cain and Victor R Lee. 2016. Measuring Electrodermal Activity to Capture Engagement in an Afterschool Maker Program. In Proceedings of the 6th Annual Conference on Creativity and Fabrication in Education (FabLearn 2016). ACM. Google ScholarDigital Library
- Steven Cantrell and Thomas J Kane. 2013. Ensuring Fair and Reliable Measures of Effective Teaching: Culminating Findings from the MET Project's Three-year Study. MET Project Research Paper.Google Scholar
- Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. 2002. SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research. Google ScholarDigital Library
- Jingjing Chen, Bin Zhu, Olle Balter, Jianliang Xu, Weiwen Zou, Anders Hedman, Rongchao Chen, and Mengdie Sang. 2017. FishBuddy: Promoting Student Engagement in Self-Paced Learning through Wearable Sensing. In In Proceedings of the IEEE International Conference on Smart Computing (SMARTCOMP 2017).Google ScholarCross Ref
- Sandra L Christenson, Amy L Reschly, and Cathy Wylie. 2012. Handbook of research on student engagement. Springer Science 8 Business Media.Google Scholar
- Corinna Cortes and Vladimir Vapnik. 1995. Support-Vector Networks. Machine learning. Google ScholarDigital Library
- Elena Di Lascio, Shkurta Gashi, Danilo Krasic, and Silvia Santini. 2017. In-classroom self-tracking for teachers and students: preliminary findings from a pilot study. In In Proceedings of the Adjunction Publication of the ACM International Joint Conference on Pervasive and Ubiquitous Computing (Ubicomp 2017). Google ScholarDigital Library
- Jennifer A Fredricks, Phyllis C Blumenfeld, and Alison H Paris. 2004. School Engagement: Potential of the Concept, State of the Evidence. Review of educational research.Google Scholar
- Jennifer A Fredricks and Wendy McColskey. 2012. The Measurement of Student Engagement: A Comparative Analysis of Various Methods and Student Self-report instruments. In Handbook of research on student engagement. Springer.Google Scholar
- Katsuya Fujii, Plivelic Marian, Dav Clark, Yoshi Okamoto, and Jun Rekimoto. 2018. Sync Class: Visualization System for In-Class Student Synchronization. In Proceedings of the 9th Augmented Human International Conference (AH 2018). Google ScholarDigital Library
- Maurizio Garbarino, Matteo Lai, Dan Bender, Rosalind W Picard, and Simone Tognetti. 2014. Empatica E3--A wearable wireless multisensor device for real-time computerized biofeedback and data acquisition. In Proceedings of the 4th EAI International Conference on Wireless Mobile Communication and Healthcare (MobiHealth 2014).Google Scholar
- Nitesh Goyal and Susan R Fussell. 2017. Intelligent Interruption Management using Electro Dermal Activity based Physiological Sensor for Collaborative Sensemaking. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT 2017). Google ScholarDigital Library
- Adrián Colomer Granero, Félix Fuentes-Hurtado, Valery Naranjo Ornedo, Jaime Guixeres Provinciale, Jose M Ausín, and Mariano Alcañiz Raya. 2016. A Comparison of Physiological Signal Analysis Techniques and Classifiers for Automatic Emotional Evaluation of Audiovisual Contents. Fontiers in computational neuroscience.Google Scholar
- Alberto Greco, Gaetano Valenza, Antonio Lanata, Enzo Pasquale Scilingo, and Luca Citi. 2016. cvxEDA: A convex optimization approach to electrodermal activity processing. IEEE Transactions on Biomedical Engineering.Google ScholarCross Ref
- Mariam Hassib, Stefan Schneegass, Philipp Eiglsperger, Niels Henze, Albrecht Schmidt, and Florian Alt. 2017. EngageMeter: A System for Implicit Audience Engagement Sensing Using Electroencephalography. In Proceedings of the Internationalf Conference on Human Factors in Computing Systems (CHI 2017). Google ScholarDigital Library
- Rui Henriques, Ana Paiva, and Claudia Antunes. 2013. Accessing Emotion Patterns from Affective Interactions Using Electrodermal Activity. In Proceedings of the International Conference on Affective Computing and Intelligent Interaction Humaine Association (ACII 2013). Google ScholarDigital Library
- Javier Hernandez, Zicheng Liu, Geoff Hulten, Dave DeBarr, Kyle Krum, and Zhengyou Zhang. 2013. Measuring the Engagement Level of TV Viewers. In Proceedings of the IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG 2013).Google ScholarCross Ref
- Javier Hernandez, Rob R Morris, and Rosalind W Picard. 2011. Call Center Stress Recognition with Person-specific Models. In Proceedings of the International Conference on Affective Computing and Intelligent Interaction (ACII 2011). Google ScholarDigital Library
- Javier Hernandez, Ivan Riobo, Agata Rozga, Gregory D Abowd, and Rosalind W Picard. 2014. Using Electrodermal Activity to Recognize Ease of Engagement in Children During Social Interactions. In Proceedings of the ACM Joint Conference on Pervasive and Ubiquitous Computing (Ubicomp 2014). Google ScholarDigital Library
- Marian L. Houser and Caroline Waldbuesser. 2017. Emotional Contagion in the Classroom: The Impact of Teacher Satisfaction and Confirmation on Perceptions of Student Nonverbal Classroom Behavior. College Teaching.Google Scholar
- Nathalie Japkowicz and Mohak Shah. 2011. Evaluating Learning Algorithms: A Classification Perspective. Cambridge University Press. Google ScholarDigital Library
- Natasha Jaques, Sara Taylor, Asaph Azaria, Asma Ghandeharioun, Akane Sano, and Rosalind Picard. 2015. Predicting Students' Happiness from Physiology, Phone, Mobility, and Behavioral Data. In Proceedings of the International Conference on Affective Computing and Intelligent Interaction (ACII 2015). Google ScholarDigital Library
- Celine Latulipe, Erin A Carroll, and Danielle Lottridge. 2011. Love, Hate, Arousal and Engagement: Exploring Audience Responses to Performing Arts. In Proceedings of the International Conference on Human Factors in Computing Systems (CHI 2011). Google ScholarDigital Library
- Victor R Lee, Joel R Drake, and Jeffrey L Thayne. 2016. Appropriating Quantified Self Technologies to Support Elementary Statistical Teaching and Learning. IEEE Transactions on Learning Technologies. Google ScholarDigital Library
- Jessica Lin, Eamonn Keogh, Li Wei, and Stefano Lonardi. 2007. Experiencing SAX: a Novel Symbolic Representation of Time Series. Data Mining and knowledge discovery. Google ScholarDigital Library
- David T Lykken and Peter H Venables. 1971. Direct Measurement of Skin Conductance: A Proposal for Standardization. Psychophysiology.Google Scholar
- Carl D Marci, Jacob Ham, Erin Moran, and Scott P Orr. 2007. Physiologic Correlates of Perceived Therapist Empathy and Social-Emotional Process During Psychotherapy. The Journal of nervous and mental disease.Google ScholarCross Ref
- João Maroco, Ana Lúcia Maroco, Juliana Alvares Duarte Bonini Campos, and Jennifer A Fredricks. 2016. University Student's Engagement: Development of the University Student Engagement Inventory (USEI). Psicologia: Reflexão e Crítica.Google Scholar
- Claudio Martella, Ekin Gedik, Laura Cabrera-Quiros, Gwenn Englebienne, and Hayley Hung. 2015. How Was It?: Exploiting Smartphone Sensing to Measure Implicit Audience Responses to Live Performances. In Proceedings of the 23rd ACM International Conference on Multimedia (ICME 2015). Google ScholarDigital Library
- Akhil Mathur, Nicholas D Lane, and Fahim Kawsar. 2016. Engagement-Aware Computing: Modelling User Engagement from Mobile Contexts. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing (Ubicomp 2016). Google ScholarDigital Library
- Karen S. McNeal, Jacob M. Spry, Ritayan Mitra, and Jamie L. Tipton. 2014. Measuring Student Engagement, Knowledge, and Perceptions of Climate Change in an Introductory Environmental Geology Course. Journal of Geoscience Education.Google ScholarCross Ref
- Abhinav Mehrotra, Veljko Pejovic, Jo Vermeulen, Robert Hendley, and Mirco Musolesi. 2016. My Phone and Me: Understanding People's Receptivity to Mobile Notifications. In Proceedings of the International Conference on Human Factors in Computing Systems (CHI 2016). Google ScholarDigital Library
- Regalena Melrose. 2006. Why Students Underachieve: What Educators and Parents Can Do About It. R8L Education.Google Scholar
- Hamed Monkaresi, Nigel Bosch, Rafael A. Calvo, and Sydney K. D'Mello. 2017. Automated Detection of Engagement Using Video-Based Estimation of Facial Expressions and Heart Rate. IEEE Transactions on Affective Computing. Google ScholarDigital Library
- Timothy P Mottet and Steven A Beebe. 2000. Emotional Contagion in the Classroom: An Examination of How Teacher and Student Emotions Are Related. ERIC.Google Scholar
- Kasia Muldner, Michael Wixon, Dovan Rai, Winslow Burleson, Beverly Woolf, and Ivon Arroyo. 2015. Exploring the Impact of a Learning Dashboard on Student Affect. In Proceedings of the International Conference on Artificial Intelligence in Education (AIED 2015).Google ScholarCross Ref
- Andreas C Müller and Sarah Guido. 2016. Introduction to Machine Learning with Python: a Guide for Data Scientists. O'Reilly Media, Inc.Google Scholar
- Richard V Palumbo, Marisa E Marraccini, Lisa L Weyandt, Oliver Wilder-Smith, Heather A McGee, Siwei Liu, and Matthew S Goodwin. 2017. Interpersonal Autonomic Physiology: A Systematic Review of the Literature. Personality and Social Psychology Review.Google Scholar
- Phuong Pham and Jingtao Wang. 2015. AttentiveLearner: Improving Mobile MOOC Learning via Implicit Heart Rate Tracking. In Proceedings of the International Conference on Artificial Intelligence in Education (AIED 2015).Google ScholarCross Ref
- Rosalind Picard. 1997. Affective computing. Vol. 252. MIT press Cambridge. Google ScholarDigital Library
- Martin Pielot, Bruno Cardoso, Kleomenis Katevas, Joan Serrà, Aleksandar Matic, and Nuria Oliver. 2017. Beyond Interruptibility: Predicting Opportune Moments to Engage Mobile Phone Users. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT 2017). Google ScholarDigital Library
- Ming-Zher Poh, Nicholas C Swenson, and Rosalind W Picard. 2010. A wearable Sensor for Unobtrusive, Long-term Assessment of Electrodermal Activity. IEEE Transactions on Biomedical Engineering.Google Scholar
- John P Pollak, Phil Adams, and Geri Gay. 2011. PAM: a Photographic Affect Meter for Frequent, in Situ Measurement of Affect. In Proceedings of the ACM International Conference on Human Factors in Computing Systems (CHI 2011). Google ScholarDigital Library
- Kaśka Porayska-Pomsta, Manolis Mavrikis, Sidney D'Mello, Cristina Conati, and Ryan SJd Baker. 2013. Knowledge Elicitation Methods for Affect Modelling in Education. International Journal of Artificial Intelligence in Education. Google ScholarDigital Library
- Pavel Pudil, Jana Novovičová, and Josef Kittler. 1994. Floating Search Methods in Feature Selection. Pattern recognition letters. Google ScholarDigital Library
- Samara Ruiz, Sven Charleer, Maite Urretavizcaya, Joris Klerkx, Isabel Fernández-Castro, and Erik Duval. 2016. Supporting Learning by Considering Emotions: Tracking and Visualization a Case Study. In Proceedings of the 6th ACM International Conference on Learning Analytics 8 Knowledge (LAK 2016). Google ScholarDigital Library
- Akane Sano and Rosalind W Picard. 2013. Stress Recognition Using Wearable Sensors and Mobile Phones. In Proceedings of the International Conference on Affective Computing and Intelligent Interaction (ACII 2013). Google ScholarDigital Library
- Yutaka Sasaki. 2007. The Truth of the F-measure. Teach Tutor mater.Google Scholar
- Martin EP Seligman. 2012. Flourish: A visionary new understanding of happiness and wellbeing. Simon and Schuster.Google Scholar
- Fernando Silveira, Brian Eriksson, Anmol Sheth, and Adam Sheppard. 2013. Predicting Audience Responses to Movie Content from Electrodermal Activity Signals. In Proceedings of the ACM international Joint Conference on Pervasive and Ubiquitous Computing (Ubicomp 2013). Google ScholarDigital Library
- Petr Slovák, Paul Tennent, Stuart Reeves, and Geraldine Fitzpatrick. 2014. Exploring Skin Conductance Synchronisation in Everyday Interactions. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational (NordiCHI 2014). Google ScholarDigital Library
- Vasileios Triglianos, Cesare Pautasso, Alessandro Bozzon, and Claudia Hauff. 2016. Inferring Student Attention with ASQ. In Proceedings of the European Conference on Technology Enhanced Learning (ECTEL 2016).Google ScholarCross Ref
- Chen Wang and Pablo Cesar. 2014. Do We React in the Same Manner?: Comparing GSR Patterns Across Scenarios. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational (NordiCHI 2014). Google ScholarDigital Library
- Chen Wang and Pablo Cesar. 2015. Physiological Measurement on Students' Engagement in a Distributed Learning Environment. In Proceedings of the 2nd International Conference on Physiological Computing Systems (PhyCS 2015). Google ScholarDigital Library
- Rui Wang, Weichen Wang, Min SH Aung, Dror Ben-Zeev, Rachel Brian, Andrew T Campbell, Tanzeem Choudhury, Marta Hauser, John Kane, Emily A Scherer, et al. 2017. Predicting Symptom Trajectories of Schizophrenia using Mobile Sensing. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT 2017). Google ScholarDigital Library
- J. Whitehill, Z. Serpell, Y-C Lin, A. Foster, and J. Movellan. 2014. The Faces of Engagement: Automatic Recognition of Student Engagement from Facial Expressions. In IEEE Transactions on Affective Computing.Google Scholar
- Yuning Zhang, Maysam Haghdan, and Kevin S Xu. 2017. Unsupervised Motion Artifact Detection in Wrist-measured Electrodermal Activity Data. arXiv preprint arXiv:1707.08287 (2017). Google ScholarDigital Library
Index Terms
- Unobtrusive Assessment of Students' Emotional Engagement during Lectures Using Electrodermal Activity Sensors
Recommendations
Elderly daily activity habits or lifestyle in their natural environments
PETRA '11: Proceedings of the 4th International Conference on PErvasive Technologies Related to Assistive EnvironmentsA research and development innovation project partly funded by the French company EDF was conducted for the advancement of smart homes. The aim is to help elderly to live at home in safe conditions. The experiments were carried out in a long-term ...
Effects of Learning Analytics on Students' Self-Regulated Learning in Flipped Classroom
The present article is aimed at analyzing the effects of learning analytics on students' self-regulated learning in a flipped classroom. An experiment was conducted with 96 engineering students, enrolled in a subject offered in the Flipped Classroom ...
Using wearable activity type detection to improve physical activity energy expenditure estimation
UbiComp '10: Proceedings of the 12th ACM international conference on Ubiquitous computingAccurate, real-time measurement of energy expended during everyday activities would enable development of novel health monitoring and wellness technologies. A technique using three miniature wearable accelerometers is presented that improves upon state-...
Comments