Skip to main content
SpringerLink
Log in

Affective Computing for eHealth Using Low-Cost Remote Internet of Things-Based EMG Platform

  • Chapter
  • First Online:
Intelligent Internet of Things for Healthcare and Industry

Part of the book series: Internet of Things ((ITTCC))

Abstract

Research in emotion recognition is important in many domains, from industrial ergonomics and health diagnostics to neuromarketing and affective computing. Emotion recognition from facial expression can be performed using image processing of face photos or using directly attached sensors reading the physiological signals from the muscular system governing the facial expression. We describe our experiments with the recognition of affectively induced facial expressions using non-intrusive on-body electromyography (EMG) sensors from the MySignals biosignal acquisition platform. For induction of the affective reaction, we use three different visual stimuli (movie clips) aimed to induce neutral, positive, and negative reactions. The results from 14 subjects show that 97.6% accuracy of emotion recognition was achieved using the covariance matrix features and K-nearest neighbor (KNN) classifier.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. A.R. Javed, L.G. Fahad, A.A. Farhan, S. Abbas, G. Srivastava, R.M. Parizi, M.S. Khan, Automated cognitive health assessment in smart homes using machine learning. Sustain. Cities Soc., 65 (2021). https://doi.org/10.1016/j.scs.2020.102572

  2. D. Połap, G. Srivastava, K. Yu, Agent architecture of an intelligent medical system based on federated learning and blockchain technology. J. Inf. Secur. Appl., 58 (2021). https://doi.org/10.1016/j.jisa.2021.102748

  3. R.W. Picard, Affective Computing (MIT Press, Cambridge, MA, 1997)

    Google Scholar 

  4. A. Luneski, E. Konstantinidis, P.D. Bamidis, Affective medicine. Methods Inf. Med. 49(3), 207–218 (2010). https://doi.org/10.3414/me061

    Article  Google Scholar 

  5. J.M. Garcia-Garcia, V.M.R. Penichet, M.D. Lozano, J.E. Garrido, E.L.-C. Law, Multimodal affective computing to enhance the user experience of educational software applications. Mob. Inf. Syst., 1–10 (2018). https://doi.org/10.1155/2018/8751426

  6. F. Bevilacqua, H. Engström, P. Backlund, Game-calibrated and user-tailored remote detection of stress and boredom in games. Sensors 19, 2877 (2019)

    Article  Google Scholar 

  7. C. Filippini, E. Spadolini, D. Cardone, A. Merla, Thermal imaging based affective computing for educational robot. Proceedings 27, 27 (2019)

    Google Scholar 

  8. G. Liaudanskaitė, G. Saulytė, J. Jakutavičius, E. Vaičiukynaitė, L. Zailskaitė-Jakštė, R. Damaševičius, Analysis of affective and gender factors in image comprehension of visual advertisement. Adv. Intell. Syst. Comput. 764, 1–11 (2019)

    Google Scholar 

  9. P. Ekman, An argument for basic emotions. Cognit. Emot. 6(3–4), 169–200 (1992)

    Article  Google Scholar 

  10. P.J. Lang, The emotion probe: Studies of motivation and attention. Am. Psychol. 50(5), 372 (1995)

    Article  Google Scholar 

  11. I.B. Mauss, M.D. Robinson, Measures of emotion: A review. Cognit. Emot. 23(2), 209–237 (2009)

    Article  Google Scholar 

  12. A. Ree, I. Morrison, H. Olausson, U. Sailer, M. Heilig, L.M. Mayo, Using facial electromyography to assess facial muscle reactions to experienced and observed affective touch in humans. J. Vis. Exp. 145 (2019). https://doi.org/10.3791/59228

  13. E. Coutinho, K. Gentsch, J. Van Peer, K.R. Scherer, B.W. Schuller, Evidence of emotion-antecedent appraisal checks in electroencephalography and facial electromyography. PLoS One 13(1) (2018). https://doi.org/10.1371/journal.pone.0189367

  14. A. Sahayadhas, K. Sundaraj, M. Murugappan, Detecting driver drowsiness based on sensors: A review. Sensors 12(12), 16937–16953 (2012). https://doi.org/10.3390/s121216937

    Article  Google Scholar 

  15. J. Šalkevicius, R. Damaševičius, R. Maskeliunas, I. Laukienė, Anxiety level recognition for virtual reality therapy system using physiological signals. Electronics 8, 1039 (2019)

    Article  Google Scholar 

  16. J. Lee, Y. Kagamihara, S. Kakei, Quantitative evaluation of movement disorders in neurological diseases based on EMG signals. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2008, 181–184 (2008). https://doi.org/10.1109/IEMBS.2008.4649120

    Article  Google Scholar 

  17. J. Wijsman, B. Grundlehner, H. Liu, J. Penders, H. Hermens, Wearable physiological sensors reflect mental stress state in office-like situations, in 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, 2013, pp. 600–605

    Google Scholar 

  18. V. Raudonis, R. Maskeliūnas, K. Stankevičius, R. Damaševičius, Gender, age, colour, position and stress: How they influence attention at workplace?, in International Conference on Computational Science and Its Applications, ICCSA 2017, Trieste, Italy, July 3–6 2017, pp. 248–264. https://doi.org/10.1007/978-3-319-62404-4_19

  19. R. Maskeliūnas, T. Blažauskas, R. Damaševičius, Depression behavior detection model based on participation in serious games, in International Joint Conference on Rough Sets, IJCRS 2017, 3–7 July 2017, Olsztyn, Poland, 2017, pp. 423–434

    Google Scholar 

  20. S. Li, G. Walters, J. Packer, N. Scott, Using skin conductance and facial electromyography to measure emotional responses to tourism advertising. Curr. Issue Tour., 1–23 (2016). https://doi.org/10.1080/13683500.2016.1223023

  21. M. Vasiljevas, R. Turcinas, R. Damasevicius, EMG speller with adaptive stimulus rate and dictionary support. in Federated Conference on Computer Science and Information Systems, FedCSIS 2014, Warsaw, Poland, 2014, pp. 227–234. https://doi.org/10.15439/2014F338

  22. M. Vasiljevas, R. Turčinas, R. Damaševičius (2014). Development of EMG-based speller, in ACM International Conference Proceeding Series, 10–12 September 2014

    Google Scholar 

  23. R. Damaševičius, M. Vasiljevas, T. Šumskas, Development of a concept-based EMG-based speller. DYNA (Colombia) 82(193), 170–179 (2015)

    Article  Google Scholar 

  24. K. Erin, B. Boru, Real time control of industrial robot arm with EMG and gyroscope data. Sakarya Univ. J. Sci., 1 (2018). https://doi.org/10.16984/saufenbilder.339232

  25. L. Yang, B. Kang, T. Wang, T. Zhao, An ergonomic study of Firefighters’ postural comfort evaluation based on EMG method. Adv. Intell. Syst. Comput. 58(1), 2310–2314 (2014)

    Google Scholar 

  26. V. Balasubramanian, G.S. Prasad, An EMG-based ergonomic evaluation of manual bar bending. Int. J. Ind. Syst. Eng. 2(3), 299 (2007)

    Google Scholar 

  27. K.R. Balasubramanian, Fatigue evaluation in manual handling using surface EMG and ergonomic Design of Trolley. Ergon. Int. J. 2(3) (2018)

    Google Scholar 

  28. M. Motamedzade, D. Afshari, A. Soltanian, The impact of ergonomically designed workstations on shoulder EMG activity during carpet weaving. Health Promot. Perspect. 4(2), 144–150 (2014). https://doi.org/10.5681/hpp.2014.019

    Article  Google Scholar 

  29. J. Kaiser, M.M. Crespo-Llado, C. Turati, E. Geangu, The development of spontaneous facial responses to others’ emotions in infancy: An EMG study. Sci. Rep. 7(1) (2017). https://doi.org/10.1038/s41598-017-17556-y

  30. M. Perusqua-Hernandez, M. Hirokawa, K. Suzuki, A wearable device for fast and subtle spontaneous smile recognition. IEEE Trans. Affect. Comput. 8(4), 522–533

    Google Scholar 

  31. M. Riehle, T.M. Lincoln, Social consequences of subclinical negative symptoms: An EMG study of facial expressions within a social interaction. J. Behav. Ther. Exp. Psychiatry 55, 90–98 (2017). https://doi.org/10.1016/j.jbtep.2017.01.003

    Article  Google Scholar 

  32. Guendil, Z., Lachiri, Z., Maaoui, C., Pruski, A. (2016). Emotion recognition from physiological signals using fusion of wavelet based features, in Proceedings of 7th International Conference on Modelling, Identification and Control, ICMIC 2015, https://doi.org/10.1109/ICMIC.2015.7409485

  33. U. Hess, R. Arslan, H. Mauersberger, C. Blaison, M. Dufner, J.J.A. Denissen, M. Ziegler, Reliability of surface facial electromyography. Psychophysiology 54(1), 12–23 (2017)

    Article  Google Scholar 

  34. M.D. Latham, N. Cook, J.G. Simmons, M.L. Byrne, J.W.L. Kettle, O. Schwartz, N.B. Allen, Physiological correlates of emotional reactivity and regulation in early adolescents. Biol. Psychol. (2017). https://doi.org/10.1016/j.biopsycho.2017.07.018

  35. M.S. Perez-Rosero, B. Rezaei, M. Akcakaya, S. Ostadabbas Decoding emotional experiences through physiological signal processing, in Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2017, New Orleans, LA, USA, March 5–9, 2017, pp. 881–885. https://doi.org/10.1109/ICASSP.2017.7952282

  36. D. Thompson, I.G. Mackenzie, H. Leuthold, R. Filik, Emotional responses to irony and emoticons in written language: Evidence from EDA and facial EMG. Psychophysiology 53(7), 1054–1062 (2016). https://doi.org/10.1111/psyp.12642

    Article  Google Scholar 

  37. V. Kehri, R. Ingle, S. Patil, R.N. Awale, Analysis of facial EMG signal for emotion recognition using wavelet packet transform and SVM, in Advances in Intelligent Systems and Computing, (Springer, Singapore, 2019), pp. 247–257

    Google Scholar 

  38. S. Orguc, H.S. Khurana, K.M.Stankovic, H.S. Leel, A.P. Chandrakasan, EMG-based real time facial gesture recognition for stress monitoring, in Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2018, pp. 2651–2654

    Google Scholar 

  39. M.M.P. Lajante, O. Droulers, D. Amarantini, How reliable are “state-of-the-art” facial EMG processing methods?: Guidelines for improving the assessment of emotional valence in advertising research. J. Advert. Res. 57(1), 28–37 (2017)

    Article  Google Scholar 

  40. Y. Guo, Y. Xia, J. Wang, H. Yu, R.-C. Chen, Real-time facial affective computing on Mobile devices. Sensors 20, 870 (2020)

    Article  Google Scholar 

  41. M. Gazzoni, B. Afsharipour, R. Merletti, Surface EMG in ergonomics and occupational medicine, in Surface Electromyography?: Physiology, Engineering, and Applications, (Wiley, London, 2016), pp. 361–391

    Chapter  Google Scholar 

  42. J. Perdiz, G. Pires, U.J. Nunes, Emotional state detection based on EMG and EOG biosignals: A short survey, in 5th Portuguese Meeting on Bioengineering, ENBENG 2017, 16–18 February 2017, Coimbra, Portugal, https://doi.org/10.1109/ENBENG.2017.7889451

  43. A.B. Jani, R. Bagree, A.K. Roy, Design of a low-power, low-cost ECG EMG sensor for wearable biometric and medical application. 2017 IEEE Sensors, 2017, https://doi.org/10.1109/icsens.2017.8234427

  44. M. Jiang, T.N. Gia, A. Anzanpour, A.-M. Rahmani, T. Westerlund, S. Salantera, P. Liljeberg, H. Tenhunen, IoT-based remote facial expression monitoring system with sEMG signal, in 2016 IEEE Sensors Applications Symposium (SAS), 2016

    Google Scholar 

  45. E. De Buyser, E. De ConinckE., B. Dhoedt, P. Simoens, Exploring the potential of combining smart glasses and consumer-grade EEG/EMG headsets for controlling IoT appliances in the smart home, in 2nd IET International Conference on Technologies for Active and Assisted Living (TechAAL 2016), 2016, https://doi.org/10.1049/ic.2016.0053

  46. R. Rastogi, D.K. Chaturvedi, S. Satya, N. Arora, M. Gupta, V. Yadav, S. Chauhan, P. Sharma, Chronic TTH analysis by EMG and GSR biofeedback on various modes and various medical symptoms using IoT, in Big Data Analytics for Intelligent Healthcare Management, (Elsevier, 2019), pp. 87–150. https://doi.org/10.1016/b978-0-12-818146-1.00005-2

    Chapter  Google Scholar 

  47. M. Zanghieri, S. Benatti, A. Burrello, V. Kartsch, F. Conti, L. Benini, Robust real-time embedded EMG recognition framework using temporal convolutional networks on a multicore IoT processor. IEEE Trans. Biomed. Circuits Syst. 14(2), 244–256 (2020). https://doi.org/10.1109/tbcas.2019.2959160

    Article  Google Scholar 

  48. M. Ali, A.A. Ali, A.-E. Taha, I.B. Dhaou, T.N. Gia, Intelligent autonomous elderly patient home monitoring system, in ICC 2019–2019 IEEE International Conference on Communications (ICC). ICC, 2019. https://doi.org/10.1109/icc.2019.8761204

  49. Libelium MySignals. http://www.my-signals.com/. Accessed: 2/2/2021

  50. E. Lopez-Larraz, O.M. Mozos, , J.M. Antelis, J. Minguez, Syllable-based speech recognition using EMG, in 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, 2010, pp. 4699–4702. https://doi.org/10.1109/IEMBS.2010.5626426

  51. van Boxtel A., Facial EMG as a tool for inferring affective states, in Proceedings of measuring behavior (Noldus Information Technology, Wageningen, 2010), pp. 104–108

    Google Scholar 

  52. R. Damasevicius, M. Vasiljevas, I. Martisius, V. Jusas, D. Birvinskas, M. Wozniak, BoostEMD: An extension of EMD method and its application for denoising of EMG signals. Elektronika ir Elektrotechnika 21(6), 57–61 (2015). https://doi.org/10.5755/j01.eee.21.6.13763

    Article  Google Scholar 

  53. N.E. Huang, Z. Shen, S.R. Long, M.C. Wu, H.H. Shih, Q. Zheng, H.H. Liu, The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proc R Soc Lond A Math Phys Eng Sci 454(1971), 903–995 (1998). https://doi.org/10.1098/rspa.1998.0193

    Article  MathSciNet  MATH  Google Scholar 

  54. M.W. Fagerland, S. Lydersen, P. Laake, The McNemar test for binary matched-pairs data: Mid-p and asymptotic are better than exact conditional. BMC Med. Res. Methodol. 13, 91 (2013). https://doi.org/10.1186/1471-2288-13-91

    Article  Google Scholar 

  55. P. Barros, N. Churamani, E. Lakomkin, H. Siqueira, A. Sutherland, S. Wermter, The OMG-emotion behavior dataset. arXiv preprint arXiv:1803.05434, 2018

    Google Scholar 

  56. K.J. Varcin, S.A. Grainger, J.L. Richmond, P.E. Bailey, J.D. Henry, A role for affectivity in rapid facial mimicry: An electromyographic study. Soc. Neurosci. 14(5), 608–617 (2019). https://doi.org/10.1080/17470919.2018.1564694

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Software Engineering, Kaunas University of Technology, Kaunas, Lithuania

    Žygintas Tamulis, Mindaugas Vasiljevas & Robertas Damaševičius

  2. Department of Applied Informatics, Vytautas Magnus University, Kaunas, Lithuania

    Rytis Maskeliunas

  3. Faculty of Applied Mathematics, Silesian University of Technology, Gliwice, Poland

    Rytis Maskeliunas

  4. Department of Electrical and Information Engineering, Covenant University, Ota, Nigeria

    Sanjay Misra

Authors
  1. Žygintas Tamulis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mindaugas Vasiljevas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Robertas Damaševičius
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rytis Maskeliunas
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sanjay Misra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Robertas Damaševičius .

Editor information

Editors and Affiliations

  1. Vanderbilt University, Nashville, TN, USA

    Uttam Ghosh

  2. Electronics & Communication Engineering, Birla Institute of Technology, Jharkhand, India

    Chinmay Chakraborty

  3. Information and Communication Technology, University of Malta, Msida, Malta

    Lalit Garg

  4. Mathematics and Computer Science, Brandon University, Brandon, MB, Canada

    Gautam Srivastava

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Tamulis, Ž., Vasiljevas, M., Damaševičius, R., Maskeliunas, R., Misra, S. (2022). Affective Computing for eHealth Using Low-Cost Remote Internet of Things-Based EMG Platform. In: Ghosh, U., Chakraborty, C., Garg, L., Srivastava, G. (eds) Intelligent Internet of Things for Healthcare and Industry. Internet of Things. Springer, Cham. https://doi.org/10.1007/978-3-030-81473-1_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-81473-1_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-81472-4

  • Online ISBN: 978-3-030-81473-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation