Abstract
This work provides an approach to monitoring the activities of people with dementia. Classification of daily activities helps automate caregivers’ documentation of activities and can provide information about disease progression and disease-related changes. The prototype developed combines smartwatch technology with a neural network to classify activities in real time. A smartwatch offers the opportunity to integrate sensor technology into a patient’s daily life without disruption.
To identify promising combinations of sensor data, accelerometer, gyroscope, gravity, and attitude data are sampled 20 Hz (Hz) and sent to a recurrent neural network called Long Short-Term Memory (LSTM) for real-time classification. In this work, we systematically compare how well an LSTM works in combination with different sensor combinations in detecting different activities. Using triaxial user acceleration, gravity, position, and gyroscope data, the trained model achieves over 90% accuracy.
The implemented real-time classification provides a direct statement at which time and with which probability one of the four activities was carried out.
This approach performs significantly better than previous tests using classification algorithms and also stands out from similar approaches in the literature. We achieve flexible, location-independent classification of very similar activities based on smartwatch sensor data. In doing so, the classification of the novel prototype is performed in real-time. This also provides room for interpretation of how the knowledge gained can be used to detect motor skills in the care of patients with neurological diseases.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alpert, J.M., et al.: Secondary care provider attitudes towards patient generated health data from smartwatches. NPJ Digit. Med. 3(3) (2020). https://doi.org/10.1038/s41746-020-0236-4
Apple Inc.: Apple watch series 6 - technical specifications (2021). https://support.apple.com/kb/SP826
Apple Inc.: Cmmotionmanager | apple developer documentation (2021). https://developer.apple.com/documentation/coremotion/cmmotionmanager. cMMotionManager
Ashry, S., Ogawa, T., Gomaa, W.: CHARM-deep: continuous human activity recognition model based on deep neural network using IMU sensors of smartwatch. IEEE Sens. J. 20(15), 8757–8770 (2020)
Askari, S.I., Huldtgren, A., IJsselsteijn, W.: Wear it or fear it - exploration of drivers & barriers in smartwatch acceptance by senior citizens. In: Bamidis, P., Ziefle, M., Maciaszek, L. (eds.) Proceedings of the 4th International Conference on Information and Communication Technologies for Ageing Well and e-Health, pp. 26–36. No. 11 in 1, SCITEPRESS - Science and Technology Publications, Funchal, Portugal (2018). https://doi.org/10.5220/0006673000260036
Bhattacharjee, S., Kishore, S., Swetapadma, A.: A comparative study of supervised learning techniques for human activity monitoring using smart sensors. In: 2018 Second International Conference on Advances in Electronics, Computers and Communications (ICAECC), pp. 1–4 (2018). https://doi.org/10.1109/ICAECC.2018.8479436
Bienhaus, D.: Smartwatch und Wearables im Gesundheitsbereich: Grundlagen und Anwendungen. Gesellschaft für Informatik e.V, Bonn (2016)
Brezmes, T., Gorricho, J.-L., Cotrina, J.: Activity recognition from accelerometer data on a mobile phone. In: Omatu, S., et al. (eds.) IWANN 2009. LNCS, vol. 5518, pp. 796–799. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-02481-8_120
Bräunel, B., Häber, A.: Wearables – Zukunftstechnologie für die geriatrische Pflege? In: Pfannstiel, M.A., Da-Cruz, P., Mehlich, H. (eds.) Digitale Transformation von Dienstleistungen im Gesundheitswesen VI, pp. 311–332. Springer, Wiesbaden (2019). https://doi.org/10.1007/978-3-658-25461-2_16
Cole, C.A., Anshari, D., Lambert, V., Thrasher, J.F., Valafar, H.: Detecting smoking events using accelerometer data collected via smartwatch technology: Validation study. JMIR mHealth and uHealth 5(12), e189 (2017)
Dadafsha, M.: Accelerometer and gyroscopes sensors: operation, sensing, and applications (2014). https://pdfserv.maximintegrated.com/en/an/AN5830.pdf
England, K., Azzopardi-Muscat, N.: Demographic trends and public health in Europe. Eur. J. Public Health 27(suppl._4), 9–13 (2017). https://doi.org/10.1093/eurpub/ckx159
Igor Lopes de Faria, V.V.: A comparative study on fitness activity recognition. In: Proceedings of the 24th Brazilian Symposium on Multimedia and the Web, WebMedia 2018, pp. 327–330. Association for Computing Machinery, New York (2018). https://doi.org/10.1145/3243082.3267452
Guo, M., Wang, Z.: Segmentation and recognition of human motion sequences using wearable inertial sensors. Multimedia Tools Appl. 77(16), 21201–21220 (2018). https://doi.org/10.1007/s11042-017-5573-1
Henriksen, A., et al.: Using fitness trackers and smartwatches to measure physical activity in research: analysis of consumer wrist-worn wearables. J. Med. Internet Res. 20(3), e110 (2018)
Hering, E., Schönfelder, G. (eds.): Sensoren in Wissenschaft und Technik: Funktionsweise und Einsatzgebiete, IArC monographs on the evaluation of carcinogenic risks to humans, 1st edn, vol. 102. Vieweg+Teubner Verlag, Wiesbaden (2012). https://doi.org/10.1007/978-3-8348-8635-4. https://monographs.iarc.fr/wp-content/uploads/2018/06/mono102.pdf
Hochreiter, S.: The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int. J. Uncertain. Fuzziness Knowl.-Based Syst. 6, 107–116 (1998). https://doi.org/10.1142/S0218488598000094
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Laurent, C., Pereyra, G., Brakel, P., Zhang, Y., Bengio, Y.: Batch normalized recurrent neural networks. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Lujiazui, China, pp. 2657–2661. IEEE (2016). https://doi.org/10.1109/icassp.2016.7472159
Lipton, Z.C., Berkowitz, J., Elkan, C.: A critical review of recurrent neural networks for sequence learning. arXiv abs/1506.00019 (2015)
Liu, Y., Pharr, M., Salvatore, G.A.: Lab-on-skin: a review of flexible and stretchable electronics for wearable health monitoring. ACS Nano 11(10), 9614–9635 (2017)
Mauldin, T.R., Canby, M.E., Metsis, V., Ngu, A.H.H., Rivera, C.C.: Smartfall: a smartwatch-based fall detection system using deep learning. Sensors 18(10) (2018). https://doi.org/10.3390/s18103363. https://www.mdpi.com/1424-8220/18/10/3363
Mekruksavanich, S., Jitpattanakul, A., Youplao, P., Yupapin, P.: Enhanced hand-oriented activity recognition based on smartwatch sensor data using LSTMs. Symmetry 12(9), 1570 (2020)
Nadal, C., et al.: Integration of a smartwatch within an internet-delivered intervention for depression: protocol for a feasibility randomized controlled trial on acceptance. Contemp. Clin. Trials 103, 106323 (2021). https://doi.org/10.1016/j.cct.2021.106323. https://www.sciencedirect.com/science/article/pii/S1551714421000598
Oluwalade, B., Neela, S., Wawira, J., Adejumo, T., Purkayastha, S.: Human activity recognition using deep learning models on smartphones and smartwatches sensor data. In: Proceedings of the 14th International Joint Conference on Biomedical Engineering Systems and Technologies - HEALTHINF, Vienna, Austria, pp. 645–650. INSTICC, SciTePress (2021). https://doi.org/10.5220/0010325906450650
Prechtl, A.: Zeit. raum. bewegung. In: Prechtl, A. (ed.) Vorlesungen über die Grundlagen der Elektrotechnik, pp. 1–14. Springer, Vienna (1994). https://doi.org/10.1007/978-3-7091-3833-5_1
Shoaib, M., Bosch, S., Scholten, H., Havinga, P., Incel, O.: Towards detection of bad habits by fusing smartphone and smartwatch sensors. In: 2015 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops), PerCom Workshops, pp. 591–596. IEEE (2015). https://doi.org/10.1109/PERCOMW.2015.7134104
Staab, S., Luderschmidt, J., Martin, L.: Recognition of usual similar activities of dementia patients via smartwatches using supervised learning. In: International Conference on Progress in Informatics and Computing, Sessions in Shanghai and Tampere. IEEE (2021). https://doi.org/10.1109/PIC53636.2021.9687025. pIC
Staab, S., Martin, L.: Informationsplattform INFODOQ. In: 56. Jahrestagung der Deutschen Gesellschaft für Sozialmedizin und Prävention, Leipzig Germany, vol. 1 (2021). dGSMP
Statista GmbH: Ganz allgemein, warum nutzen sie ihre smartwatch? (2016). https://de.statista.com/statistik/daten/studie/588913/umfrage/umfrage-zu-nutzungsgruenden-einer-smartwatch-in-deutschland/
Fritz, D., et al.: Der elektronische kompaß. Design & Elektronik Sensortechnik, pp. 28–30 (1995)
Vijayan, V., Connolly, J.P., Condell, J., McKelvey, N., Gardiner, P.: Review of wearable devices and data collection considerations for connected health. Sensors 21(16) (2021). https://doi.org/10.3390/s21165589. https://www.mdpi.com/1424-8220/21/16/5589
Weiss, G.M., Timko, J.L., Gallagher, C.M., Yoneda, K., Schreiber, A.J.: Smartwatch-based activity recognition: a machine learning approach. In: 2016 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), Las Vegas, NV, USA, pp. 426–429. IEEE (2016). https://doi.org/10.1109/BHI.2016.7455925
Wu, J., Zhou, Z., Chen, J., Fourati, H., Li, R.: Fast complementary filter for attitude estimation using low-cost MARG sensors. IEEE Sens. J. 16(18), 6997–7007 (2016)
Xia, C., Sugiura, Y.: Wearable accelerometer optimal positions for human motion recognition. In: 2020 IEEE 2nd Global Conference on Life Sciences and Technologies (LifeTech), Kyoto, Japan, pp. 19–20. IEEE (2020). https://doi.org/10.1109/LifeTech48969.2020.1570618961
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Staab, S., Bröning, L., Luderschmidt, J., Martin, L. (2023). Live Classification of Similar Arm Motion Sequences Using Smartwatches. In: Gao, Q., Zhou, J. (eds) Human Aspects of IT for the Aged Population. HCII 2023. Lecture Notes in Computer Science, vol 14043. Springer, Cham. https://doi.org/10.1007/978-3-031-34917-1_25
Download citation
DOI: https://doi.org/10.1007/978-3-031-34917-1_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-34916-4
Online ISBN: 978-3-031-34917-1
eBook Packages: Computer ScienceComputer Science (R0)